The only known use case for this is the hash-to-curve operation where the top bit is always cleared.
But the function is public, so let's make it work as one would expect in the general case.
Also fix the comment by the way.
c_void is *not* simply `const c_void = opaque{};`. It has unique
semantics as any pointer type may coerce to `*c_void` which is not true
for an arbitrary `*opaque{}`.
Previously the fd parameter was ignored and so the result would not get
populated. Now it passes the fd pointer to the inline assembly so that
the results can be observed.
This new option sets a default libc paths file to be used for all
LibExeObjSteps. Setting LibExeObjStep.libc_file overrides this default.
This is required to allow users to cross compile projects linking system
libraries without needing to patch the build.zig.
Previous to #7082, users could overwrite PATH_MAX in the root file to support std.os.toPosixPath, permitting the "bring your own operating system" layer to implement the POSIX API for opening files. Unfortunately that is no longer the case.
This commit intends to fix what is arguably a regression from 0.7 in a way that doesn't break any code targeting 0.8.0, making it suitable to be included in a 0.8 patch release.
However in a future release that permits breaking changes, I am of the opinion that it would be beneficial to overwrite the value, even for "supported" operating systems. Same for all the other POSIX/BYOOS functions and values. However this is beyond the scope of this commit. Further discussion of this will be made into an issue in due time.
* Avoid emitting the copy_file_range symbol at all to prevent link-time
errors.
* Fix a bug in the check logic, the has_copy_file_range_syscall was
set to the wrong value in case of ENOSYS
* If link_libc is true don't fall-back to the raw syscall approach,
there's no policy about what to do in this case but let's follow what
the other impls do.
Fixes#9146
`return` statements use a new function `nodeMayEvalToError` which does
some basic checks on the AST node to return never, always, or maybe.
Depending on this result, AstGen skips the errdefers, always includes
the errdefers, or emits a conditional branch to check whether the return
value is an error that Sema will have to evaluate.
Closes#8821
Unblocks #9047
Handle linker args joined with a = like -Wl,-rpath=foo
Update existing args --major-os-version, --minor-os-version,
--major-subsystem-version and --minor-subsytem-version to work with the
new parsing.
Also handle -Wl,--script in addition to -Wl,-T
Lakemont has no x86, no MMX, no SSE and no way of handling any fp-math. In theory LLVM is able to implicitly use the soft-float emulation library calls to legalize any such operation but, given Zig's use of many non-standard features, sometimes we hit a weak spot in the X86 codegen backend.
Consider this as a work-around for this LLVM problem, fixing the problem in LLVM is not so high in my todo list as the target is pretty niche and Intel axed it in '19.
(Commit message by @LemonBoy)
NetBSD expects 2 PT_LOAD segments in a shared object, otherwise
ld.elf_so fails to load, emitting a general "not found" error.
Workaround issue by adding args `--no-rosegment` and `-znorelro`.
see #9109
Previously, Zig did not properly communicate the target CPU features for
RISC-V to clang assembler, because Clang has a different way to pass CPU
features for C code and for assembly code. This commit makes Zig pass a
RISC-V -march flag in order to communicate CPU features to Clang when
compiling assembly files.
Two bugs in the implementation ported from musl made all the complex
functions relying on ldexp return incorrect results in some cases.
Spotted in #9047
* stage1 backend allows configuring the uwtables function attr
via a flag rather than its own logic.
* stage2 defaults to enabling uwtable attr when
linking libunwind, or always on windows
* stage2 makes link_eh_frame_hdr true automatically if uwtable
attr is set to be on for zig functions
* CLI: add -funwind-tables and -fno-unwind-tables to allow the user to
override the defaults.
* hook it up to `zig cc`
closes#9046
It turns out the code was not ported correctly from C and produced wrong
results for negative input values. As a bonus fix the NaN codepath by
adding yet another missing piece of code.
Spotted in #9047
closes#9034
These options were listed under the
"Debug Options (Zig Compiler Development)" heading. Anything in this
section should be considered unstable and can be modified at any time
at any developer's discretion.
Zig has detection for when it is accidentally being called recursively
when trying to find the native libc installation. However it was not
working, resulting in a cryptic failure, because zig tried to execute
a command which had spaces in it rather than tokenizing it.
This improves the user experience of `zig cc` for systems that Zig
does not support cross-compiling for.
Closes#8960
I want the language reference to be divorced from any particular
community. Also remove the call to action since the docs are
known to be incomplete and are not the current focus of the project.
Closes#9055
When linking with -lfoo syntax, this indicates to Zig that the
dependency should either be provided by Zig, or it should be dynamically
provided by the system.
For windows-gnu targets, the search path was "foo.lib". Now it
additionally looks for "libfoo.dll.a".
Closes#7799
Add two helpers to ensure people won't ignore some edge cases such as
pointers overflowing the address space.
Also fix#8924 to some degree, the amount of unchecked alignForward is
still scary.
Revert "tests: disable i386-linux-gnu -lc target due to CI failures"
This reverts commit 97a2f4e7ae.
* std: Better handing of POLLHUP in ChildProcess
Upon hitting the EOF condition there are two main differences between
how Linux and the *BSD-derived systems behave: the former sets POLLHUP
and POLLIN and, after reading any residual data, only POLLHUP remains
set. The latter signal the EOF condition by setting both flags thus
requiring some extra checks to determine if the stream is "done".
DragonFly workaround/hack for POLLHUP is no longer required.
Closes#8969
LLVM time profiler can collect time traces and present them in a
hierarchical view. Which breakdowns the time spent in each Pass or even
IR unit. The result is also exported into a format that can be easily
visualized by the Chrome browser.
Currently this features is controlled by the following environment
variables:
- `ZIG_LLVM_TIME_TRACE_FILE` toggles this feature and specifies the
output time trace file.
- `ZIG_LLVM_TIME_TRACE_GRANULARITY` controls the time granularity in
ms (default to 500).
- hash/eql functions moved into a Context object
- *Context functions pass an explicit context
- *Adapted functions pass specialized keys and contexts
- new getPtr() function returns a pointer to value
- remove functions renamed to fetchRemove
- new remove functions return bool
- removeAssertDiscard deleted, use assert(remove(...)) instead
- Keys and values are stored in separate arrays
- Entry is now {*K, *V}, the new KV is {K, V}
- BufSet/BufMap functions renamed to match other set/map types
- fixed iterating-while-modifying bug in src/link/C.zig
`msghdr` and `msghdr_const` definitions have been added back the way
they were in std.os. std.os.sendmsg has also been modified to accept a
msghdr_const again to ensure backwards-compatibility with this PR.
Underneath the hood, std.os.sendmsg will @ptrCast the provided
msghdr_const into a std.x.os.Socket.Message.
`sockaddr_storage` definitions have been added back the way they were in
std.os, except that it now simply aliases
std.x.os.Socket.Address.Native.Storage as all of
std.x.os.Socket.Address.Native.Storage's fields are equivalent to the
fields that were previously defined for std.x.os.bits.sockaddr_storage.
std.x.os.Socket.sendMessage now no longer is a stub that aliases
std.os.sendmsg, but instead calls and handles
errors from std.os.system.sendmsg directly.
Addresses feedback to urge backwards compatibility from @andrewrk.
Cross-platform versions of msghdr, sendmsg, recvmsg, linger, and iovec
were provided based on findings from glibc, musl, and Microsoft's
documentation.
Implemented initial Reactor interface for epoll (linux) which wraps
around I/O reactor subsystems such as epoll, kqueue, select, etc. across
different platforms. The Reactor interface allows for driving async I/O
in Zig applications.
A test was added for the Reactor interface to drive a TCP
client/listener socket pair.
A greatest-common-subset of possible socket initialization flags (close
socket on exec syscalls, initialize socket to be non-blocking) were
implemented.
A test was added for using sendmsg/recvmsg syscalls across different
platforms for a TCP client/listener socket pair.
Instead require `1e9` and `0x1p9`, disallowing the trailing dot.
This change to the grammar is consistent with forbidding `1.` and `0x1.`
as float literals and ensures there is only one way to do things here.
- deprecates `std.Thread.spinLoopHint` and moves it to `std.atomic.spinLoopHint`
- added an Atomic(T) generic wrapper type which replaces atomic.Bool and atomic.Int
- in Atomic(T), selectively expose member functions depending on T and include bitwise atomic methods when T is an Integer
- added fence() and compilerFence() to std.atomic
In this file is copy+pasted WindowsSupport.h from LLVM 12.0.1-rc1.
This is so that we can patch it. The upstream sources are incorrectly
including "llvm/Config/config.h" which is a private header and thus not
available in the include files distributed with LLVM.
The patch here changes it to include "llvm/Config/config.h" instead.
Patch submitted upstream: https://reviews.llvm.org/D103370
The same entrypoint supports the following commands:
* ar
* ranlib
* dlltool
* lib
For now, our strategy is to bundle the (renamed) `main()` function of
llvm-ar, same as our strategy for `zig clang`. However, as Zig matures,
a goal will be to replace the dependency on LLVM with our own
implementation of this tool, so that it is available in builds of zig
that do not have LLVM extensions enabled.
This commit also categorizes the subcommands into categories in the
--help menu.
Before this, if a compile error occurred, it would cause the previous
value for e.g. the function scope to not get reset. If the AstGen
process continued, it would result in a violation of the data
guarantees that it relies on.
This commit takes advantage of defer to ensure the previous value is
always reset, even in the case of an error.
Closes#8920
Bitcast the pointer and operands to integer types having the same size,
working around LLVM inability to lower a LL/SC operation when the
operands have floating-point types (and are reasonably sized).
Closes#4457
Currently, stage1 runs astgen for every comptime function call,
resulting in identifier strings being allocated multiple times,
wasting memory. As a workaround until the code is adjusted to
make astgen run only once per source node, we memoize the
result into the AST.
* Rename `ir_gen_*` to `astgen_*`
- Oops, meant to do this in a separate commit. My bad.
* tokenizer: avoid using designated initializer syntax.
MSVC does not support it.
This is progress towards making Stage1Zir immutable, so that we can
avoid generating it for every comptime function call.
Also rename IrExecutableGen to Stage1Air.
* Extracts AstGen logic from ir.cpp into astgen.cpp. Reduces the
largest file of stage1 from 33,551 lines to 25,510.
* tokenizer: rework it completely to match the stage2 tokenizer logic.
They can now be maintained together; when one is changed, the other
can be changed in the same way.
- Each token now takes up 13 bytes instead of 64 bytes. The tokenizer
does not parse char literals, string literals, integer literals,
etc into meaningful data. Instead, that happens during parsing or
astgen.
- no longer store line offsets. Error messages scan source
files to find the line/column as needed (same as stage2).
- main loop: instead of checking the loop, handle a null byte
explicitly in the switch statements. This is a nice improvement
that we may want to backport to stage2.
- delete some dead tokens, artifacts of past syntax that no longer
exists.
* Parser: fix a TODO by parsing builtin functions as tokens rather than
`@` as a separate token. This is how stage2 does it.
* Remove some debugging infrastructure. These will need to be redone,
if at all, as the code migrates to match stage2.
- remove the ast_render code.
- remove the IR debugging stuff
- remove teh token printing code
This will set us up to correctly retrieve the error local index and payload index
depending on that of the multi_value's index. As from now, the error will always use
the multi_value's index, and the payload will use the following locals.
- This currently uses the multi-value feature to return both the possible error, and its payload.
- Also genAlloc and the logic to allocate the locals itself have been seperated, so we can create more locals
whenever needed, and not only when `genAlloc` is called.
It currently looks like that if the user links in a dylib using
`lib_or_exe.linkSystemLibrary`, and the linked lib doesn't have
a hardcoded path in its description load command but rather it
allows for any runtime path via `@rpath`, then it is not possible
to specify the runtime path explicitly using the build system.
fiat-crypto now generates proper types, so take advantage of that.
Add mixed subtraction and double base multiplication.
We will eventually leverage mixed addition/subtraction for fixed
base multiplication. The reason we don't right now is that
precomputing the tables at comptime would take forever.
We don't use combs for the same reason. Stage2 + less function
calls in the fiat-crypto generated code will eventually address
that.
Also make the edwards25519 code consistent with these changes.
No functional changes.
Change default behaviour to search for static archives before searching
for dynamic libraries if no flag such as `-search_paths_first` is
specified. Also, fix a bug with early break from outer loop (label
in the wrong place).
All other uses of `ws2_32.socklen_t` in windows.zig casts the value to an i32. `recvfrom` should do so as well; it currently errors out with `expected type '?*i32', found '?*u32'`.
* for some reason part of the linkable bits for the crt libraries
are split in different locations for haiku. this changeset
accomodates this situation (crtbegin_dir lookup)
Clang has a completely inconsistent CLI for its integrated assembler for
each target architecture. For x86_64, for example, it does not accept
an -mcpu parameter, and emits "warning: unused parameter". However, for
ARM, -mcpu is needed in order to properly lower assembly to machine code
instructions (see new standalone test case provided thanks to @g-w1).
This is a compromise between
b8f85a805b and
afb9f695b1.
This reverts commit afb9f695b1.
I don't think this was ever actually verified to fix the thing it
purported to, and it started causing warnings for unused command line
parameters.
- more support for linux, android, freebsd, netbsd, openbsd, dragonfly
- centralize musl utils; musl logic is no longer intertwined with csu
- fix musl compilation to build crti/crtn for full archs list
- fix openbsd to support `zig build-lib -dynamic`
- initial dragonfly linking success (with a warning)
ancillary:
- fix emutls (openbsd) tests to use `try`
This addresses the use case of quickly reporting AstGen compile errors
for a file, for use with an IDE for example.
* Rename from `zig asgen` to `zig ast-check`
* It is now a command always available; not only in debug builds.
* Give it usage text and proper CLI parsing.
* Support reading from stdin when no positional arg is provided.
* `-t` flag makes it print textual ZIR. Without this flag, it only
provides compile errors.
* Support `--color` parameter to override the tty detection
closes#8871
We've settled on the nomenclature for the artifacts the compiler
pipeline produces:
1. Tokens
2. AST (Abstract Syntax Tree)
3. ZIR (Zig Intermediate Representation)
4. AIR (Analyzed Intermediate Representation)
5. Machine Code
Renaming `ir` identifiers to `air` will come with the inevitable
air-memory-layout branch that I plan to start after the 0.8.0 release.
This commits permits passing in static archives using the system
lib flag `-la`. With this commit, `zig ld` will now look firstly for
a dynamic library (which always takes precedence), and will fall back
on `liba.a` if the dylib is not found. The static archive is searched
for in the system lib search dirs like the dylibs.
This matches the behaviour for other targets in that
```
zig build-lib math.zig -target wasm32-freestanding
```
produces now `libmath.a` while
```
zig build-lib math.zig -dynamic -target wasm32-freestanding
```
is required to create a loadable Wasm module.
When WSASocketW gets WSANOTINITIALISED, now it will lock a mutex to
safely call WSAStartup and then try again one time.
This implementation:
* Does not use recursion
* Contains a detailed doc comment explaining why things are how they are
* Is careful about which errors are surfaced in the respective error
sets. `std.os.socket` intentionally does not have "not initialised"
as one of the possible errors.
This matches the behaviour of other languages and leaves us
the ability to create actual static Wasm archives with
```
zig build-lib -static some.zig
```
which can then be combined with other Wasm object files and linked
into either a Wasm lib or executable using `wasm-ld`.
Update langref to reflect the fact we now ship WASI libc.
Rename include dir to match the convention:
from `wasm32-wasi` to `wasm-wasi-musl`
Add building stubs which will be used to build and cache WASI
libc sysroot.
The atfork handler is executed even when fork()-ing threads that have
never initialized their local csprng. Handle this case gracefully
instead of raising a runtime error.
Fixes#8841
Some libc implementations (glib) deduplicate identical hooks, others
(musl, macos) do not and blindly append them to an internal list.
Ensure there's only a single call to pthread_atfork to prevent unbounded
memory use when lots of threads/forks are used.
This allows us to differentiate between regular locals and variables that create multiple locals
on the stack such as optionals and structs.
Now `struct_a = struct_b;` works and only updates a reference, rather than update all local's values.
Also created more test cases to test against this.
- When returning within a block, we must use an explicit return opcode. For now always emit the opcode when calling return, rather than using implicit return statements.
- Also added a more comprehensive test case to test for enum values using conditions
- This adds support for enum values using field indexes
- EmitConstant's signature was changed so it's easier to recursively call it using a different type (enum -> int type).
- Implemented initial support for bitcast which for now just returns the `WValue` of the operand.
We do need to link the system libc if linking system libraries as
they may potentially be compiled against e.g. a newer glibc version
than zig can provide. However if not linking system libraries, using
the zig provided libc is more reliable as it does not depend on any
quirks of the host system or being able to invoke the system cc to
find include dirs.
Now that `@import` of a non-existent file name is a compile error, the
Zig installation needs to include files that end with test.zig. However
we still want to avoid bloat in the installation size, so we blank them
out instead of omitting them entirely.
Now AstGen of the full standard library can complete successfully on a
Zig installation rather than erroring out with file not found.
When scanDecls happens, we create stub Decl objects that
have not been semantically analyzed. When they get referenced,
they get semantically analyzed.
Before this commit, when they got unreferenced, they were completely
deleted, including deleted from the containing Namespace.
However, if the update did not cause the containing Namespace to get
deleted, for example, if `std.builtin.ExportOptions` is no longer
referenced, but `std.builtin` is still referenced, and then `ExportOptions`
gets referenced again, the Namespace would be incorrectly missing the
Decl, so we get an incorrect "no such member" error.
The solution is to, when dealing with a no longer referenced Decl
objects during an update, clear them to the state they would be in
on a fresh scanDecl, rather than completely deleting them.
Current assumed order is:
* cwd
* `self.base.options.lib_dirs`
* finally, if `self.base.options.syslibroot` is defined,
`self.base.options.lib_dirs` prefixed with syslibroot
The support is minimalistic in the sense that we only support actual
dylib files and not stubs/tbds yet, and we also don't support re-exports
just yet.
This fixes a segfault in translate-c that would previously occur when
initializing structs with unnamed bitfields, due to a failed assertion in
`transInitListExprRecord`. Unnamed bitfields do not have initializers,
so `transInitListExprRecord` erroneously assumes that `init_count` equals
the number of fields in the record.
Since attempting to initialize an opaque type is a syntax error in Zig,
we can just demote any attempts to initialize them.
A previous commit from this branch incorrectly changed the usage of
`comptime` keyword, and broke the std lib tests. This commit adds
`comptime` to a few function calls, correcting the behavior.
Conflicts:
* src/codegen/spirv.zig
* src/link/SpirV.zig
We're going to want to improve the stage2 test harness to print
the source file name when a compile error occurs otherwise std lib
contributors are going to see some confusing CI failures when they cause
stage2 AstGen compile errors.
In order for this test to pass, the host linking/start code needs to
support explicitly setting the stack size. Zig defaults to 16 MiB stack
size, which is enough to pass the test in Debug builds, however, most
operating systems do not honor the stack size we request for and give a
smaller amount.
Eventually the goal is to pass this test on all hosts.
Previously, ZIR was per-function so we could simply allocate a slice for
all ZIR instructions. However now ZIR is whole-file, so we need a sparse
mapping of ZIR to AIR instructions in order to not waste memory.
Previously, stage2 used a global decl_table for all Decl objects, keyed
by a 16-byte name hash that was hopefully unique. Now, there is a tree
of Namespace objects that own their named Decl objects.
Let mmap allocate a block of memory that's wide enough to use with
MADV_WIPEONFORK, madvise granularity is the current system page size
(using a static buffer of mem.page_size bytes would be wrong, that's the
minimum page size).
As a result, we don't zero some random chunk of memory every time we
fork the process.
Fixes#7609
There was missing incremental compilation logic in the MachO linker
code, causing test failures. With this logic ported over from the
corresponding ELF logic, tests pass again.
Previously the frontend incorrectly called freeDecl for structs, which
never got allocateDecl called for them. There was simply a missing check
for hasCodeGenBits().
* Advance line and PC prior to ending sequence in debug line program
for a fn_decl. This is equivalent to closing scope in the debugger
and without it, the debugger will not map source-to-address info
as a result will not print the source when breaking at a symbol.
* Fix debug aranges sentinels to be of the size as the actual tuple
descriptor (assuming segment selector to be ommitted). In summary,
the sentinels were 32bit 0s, whereas they ought to be 64bit 0s.
* Make naming of symbols in the binary more consistent by prefixing
each symbol name with an underscore '_'.
Conflicts:
* build.zig
* src/Compilation.zig
* src/codegen/spirv/spec.zig
* src/link/SpirV.zig
* test/stage2/darwin.zig
- this one might be problematic; start.zig looks for `main` in the
root source file, not `_main`. Not sure why there is an underscore
there in master branch.
As it stands, the backend is incomplete, and there is no active contributor,
making it dead weight.
However, anyone is free to resurrect this backend at any time.
Just like when new parse errors occur during an update, when new AstGen
errors occur during an update, we do not reveal compile errors for Decl
objects which are inside of a newly failed File. Once the File passes
AstGen successfully, it will be compared with the previously succeeded
ZIR and the saved Decl compile errors will be handled properly.
Use .bold instead of .white, the effect is the same for light-on-dark
terminals but greatly improves the readability for dark-on-light ones.
Closes#8761
Previously, compile log stored full SrcLoc info, which included absolute
AST node index. This becomes invalid after an incremental compilation.
To make it survive incremental compilation, store an offset from parent
Decl instead.
* Do not report export collision errors until the very end, because it
is possible, during an update, for a new export to be added before an
old one is semantically analyzed to be deleted. In such a case there
should be no compile error.
- Likewise we defer emitting exports until the end when we know for
sure what will happen.
* Sema: Fix not adding a Decl dependency on imported files.
* Sema: Properly add Decl dependencies for all identifier and namespace
lookups.
* After semantic analysis for a Decl, if it is still marked as
`in_progress`, change it to `dependency_failure` because if the Decl
itself failed, it would have already been changed during the call to
add the compile error.
* std: fix overflow in math.scalbn32
* std: rewrite math.scalbn to be generic
* std: support f128 in math.isNormal
* std: enable f128 tests in math.scalbn
we only effectively need 1 control bit to represent 2 special states for
the metadata (free and tombstone)
this should reduce the number of actual element equality tests, but since
it's very low already, the impact is negligible
The `1 - shift` expression was computed using small unsigned types and
then casted to i32, producing either an underflow error or an incorrect
result.
Reported by `@notviri` in #8733
There is a lot left to be desired for this tool, as currently dependencies of extensions and dependencies
of capabilities on extensions are not included:
- There is no machine-readable definition of dependencies of extensions.
- A capability may depend on either of a multitude of extensions, which as of yet cannot be properly
modelled in the target system.
Originally, I thought that the dSYM bundle has to reside side-by-side
with the binary it carries the debugging information for. However, it
turns out macOS is clever enough that it auto-searches for matching
dSYM bundle based on the embedded UUID of the binary.
To verify this, run this on your macOS:
```
mdfind "com_apple_xcode_dsym_uuids == <UUID from LC_UUID>"
```
See [here] for more info.
[here]: https://developer.apple.com/documentation/xcode/adding-identifiable-symbol-names-to-a-crash-report
* Remove the ability for GenZir parent Scope to be null. Now there is a
Top Scope at the top.
* Introduce Scope.Namespace to contain a table of decl names in order
to emit a compile error for name conflicts.
* Fix use of invalid memory when reporting compile errors by
duplicating decl names into a temporary heap allocated buffer.
* Fix memory leak in while and for loops, not cleaning up their
labeled_breaks and store_to_block_ptr_list arrays.
* Fix stage2 test cases because now the source location of redundant
comptime keyword compile errors is improved.
* Implement compile error for local variable shadowing declaration.
commitDeclDebugInfo: stop trying to write length 0 debug info - avoids
hitting an error where zig tries to move the debug info section
unnecessarily, gets confused, and reports `error.InputOutput`.
The 2 pieces of code that looked at the source and tried to compute
source line offsets is now improved to match link/Elf - use the
line/column data stored in the Decl object to skip the costly scanning
of source bytes. No need to load Zig source code, AST, or tokens, when
we have the ZIR!
* `-lc++` now implies `-lc`.
* `-lunwind` is now pulled out into a separate `link_libunwind` flag in
the frontend driver code. This allows a project to request zig to
provide libunwind even if the default situation that causes it to be
implicitly added, is not active.
* build.zig: ask for -lunwind when building the self-hosted compiler on
Linux. Otherwise we get linker errors with unresolved symbols to
libunwind.
* for cross testing stage2 tests, we use `darling shell` command
since the path to the tested binary is relative to cwd
* for the `zig test` command, we simply use `darling` since the
path to the binary is absolute
With this change, it is now possible to use zig-bootstrap as the
initial C/C++ compiler, and then build stage2 with the resultant
stage1 without the need to link any system libs!
In `Module.semaDecl`, the source location being used was double-relative
to the `Decl`, causing a crash when trying to compute byte offset for
the compile error.
Change the source location to node_offset = 0 since the scope being used
makes the source location relative to the Decl, which already has the
source node index populated.
5ac91794cc made Zig link against the
system libc when targeting the native C ABI. However this made it stop
putting libunwind.a on the linker line when it needed to sometimes,
causing undefined symbols when linking against C++ code.
5ac91794cc made Zig link against the
system libc when targeting the native C ABI. However this made it stop
putting libunwind.a on the linker line when it needed to sometimes,
causing undefined symbols when linking against C++ code.
Before this change, the attempt to save the most recent successful ZIR
code only worked in some cases; this reworks the code to be more robust,
thereby fixing a crash when running the stage2 "enums" CBE test cases.
Without this, and with `-Denable-qemu -Denable-foreign-glibc`,
i386-linux-gnu tests failed because the `-L` path passed to qemu
was "i386-linux-gnu" (nonexistent) rather than "i686-linux-gnu".
Fixes std lib and behavior tests when using these options to enable
test coverage with cross compiled glibcs.
Conflicts:
* lib/std/os/linux.zig
* lib/std/os/windows/bits.zig
* src/Module.zig
* src/Sema.zig
* test/stage2/test.zig
Mainly I wanted Jakub's new macOS code for respecting stack size, since
we now depend on it for debug builds able to pass one of the test cases
for recursive comptime function calls with `@setEvalBranchQuota`.
The conflicts were all trivial.
The CPU detection code is nearly at feature parity, we do support
detecting the native CPU on Sparc systems and macos, our ARM/AArch64
model list is quite comprehensive and so is our PPC one.
The only missing pieces are:
- ARM32 detection on Darwin hosts (I don't think anybody is planning on
running the compiler on a old-ass iPhone)
- s390x detection on Linux hosts, this can be easily added at a later
stage.
The current spanZ() function will not scan for a 0 terminator if the
type is not 0 terminated. This encourages using 0 terminated array
types to bind C arrays which hold 0 terminated strings. However, this is
a big footgun as nothing in the C type system guarantees there to be a
0 terminator at the end of the array and if there is none this becomes
Illegal Behavior in Zig.
To solve this, deprecate spanZ() and lenZ(), adding a new sliceTo()
function that always scans for the given terminator even if the type is
not sentinel terminated.
* File stores `root_decl: Decl` instead of `namespace: *Namespace`.
This maps more cleanly to the actual ownership, since the `File` does
own the root decl, but it does not directly own the `Namespace`.
* `semaFile` completes the creation of the `Decl` even when semantic
analysis fails. The `analysis` field of the `Decl` will contain the
results of semantic analysis. This prevents cleaning up of memory
still referenced by other Decl objects.
* `semaDecl` sets `Struct.zir_index` of the root struct decl, which
fixes use of undefined value in case the first update contained a ZIR
compile error.
* Compilation: iteration over the deletion_set only tries to delete the
first one, relying on Decl destroy to remove itself from the deletion
set.
* link: `freeDecl` now has to handle the possibility of freeing a Decl
that was never called with `allocateDeclIndexes`.
* `deleteDecl` recursively iterates over a Decl's Namespace sub-Decl
objects and calls `deleteDecl` on them.
- Prevents Decl objects from being destroyed when they are still in
`deletion_set`.
* Sema: fix cleanup of anonymous Decl objects when an error occurs
during semantic analysis.
* tests: update test cases for fully qualified names
This avoids causing false positive compile errors when, for example, a
file had ZIR errors, and then code tried to look up a public decl from
the failed file.
* `Module.File` now retains the most recent successful ZIR in the event
that an update causes a ZIR compile error. This way Zig does not
throw out useful semantic analysis results when an update temporarily
introduces a ZIR compile error.
* Semantic analysis of a File now unconditionally creates a Decl object
for the File. The Decl object is marked as `file_failed` in case of
File-level compile errors. This allows detecting of the File being
outdated, and dependency tracking just like any other Decl.
Decl objects need to know whether they are the owner of the Type/Value
associated with them, in order to decide whether to destroy the
associated Namespace, Fn, or Var when cleaning up.
Currently zig will always try to build its own libc and compile against
that. This of course makes sense for cross-compilation, but can cause
problems when targeting the native OS/ABI.
For example, if the system uses a newer glibc version than zig ships
zig will fall back to using the newest version it does ship. However
this causes linking system libraries to fail as they are built against a
different glibc version than the zig code is built against.
To remedy this, simply default to linking the system libc when targeting
the native OS/ABI.
It turns out that nothing in the test suite was exercising
preadv/pwritev and so the previous commits silently broke them.
Adding tests revealed readvAll and preadvAll were also broken and not
covered by any test.
While musl decided to hard-wire off_t to a 64bit quantity, glibc is much
older and defaults to 32bit offsets and offers some -64 suffixed
versions of I/O functions.
There's a weird mix-up of types: sometimes off_t is used, sometimes not,
sometimes it's defined as a signed quantity and sometimes as an unsigned
one, but we'll sort this problem later.
This commit takes advantage of the new "NameStrategy" that is exposed
in the ZIR in order to name Decls after their parent when asked to. This
makes the next failing test case pass.
which can be either parent, func, or anon. Here's the enum reproduced in
the commit message for convenience:
```zig
pub const NameStrategy = enum(u2) {
/// Use the same name as the parent declaration name.
/// e.g. `const Foo = struct {...};`.
parent,
/// Use the name of the currently executing comptime function call,
/// with the current parameters. e.g. `ArrayList(i32)`.
func,
/// Create an anonymous name for this declaration.
/// Like this: "ParentDeclName_struct_69"
anon,
};
```
With this information in the ZIR, a future commit can improve the
names of structs, unions, enums, and opaques.
In order to accomplish this, the following ZIR instruction forms were
removed and replaced with Extended op codes:
* struct_decl
* struct_decl_packed
* struct_decl_extern
* union_decl
* union_decl_packed
* union_decl_extern
* enum_decl
* enum_decl_nonexhaustive
By being extended opcodes, one more u32 is needed, however we more than
make up for it by repurposing the 16 "small" bits to provide shorter
encodings for when decls_len == 0, fields_len == 0, a source node is not
provided, etc. There tends to be no downside, and in fact sometimes
upsides, to using an extended op code when there is a need for flag
bits, which is the case for all three of these. Likewise, the container
layout can be encoded in these bits rather than into the opcode.
The following 4 ZIR instructions were added, netting a total of 4 freed
up ZIR enum tags for future use:
* opaque_decl_anon
* opaque_decl_func
* error_set_decl_anon
* error_set_decl_func
This is so that opaques and error sets can have the same name hint as
structs, enums, and unions.
`std.builtin.ContainerLayout` gets an explicit integer tag type so that
it can be used inside packed structs.
This commit also makes `Module.Namespace` use a separate set for
anonymous decls, thus allowing anonymous decls to share the same
`Decl.name` as their owner `Decl` objects.
Implement loading Winsock extensions.
Add missing Winsock extension GUID's.
Implement readVectorized() for POSIX sockets and
readVectorized() / writeVectorized() for Windows
sockets.
Inverse how mixins are used to implement platform-independent syscalls
for the std.x.os.Socket abstraction. This cleans up the API as suggested
by @komuw.
Add missing constants to ws2_32 such as SIO_BASE_HANDLE.
Made all declared constants in ws2_32 comptime_int, and fixed a few
broken constants (i.e. GUID's).
Fixed a few syscalls not having callconv(WINAPI) in ws2_32.
Fixed a typo in std.x.net.tcp.Client.
Removed unnecessary @alignCast's for socket addresses in
std.x.net.Socket.
Added a warning on using timeout methods for std.x.net.Socket.
Fixed compilation error spotted by CI in std.os that references
std.os.windows.ws2_32.SD_RECEIVE.
Renamed std.x.os.Socket.setOption()'s parameter `name: u32` to `code:
u32`.
Socket I/O methods such as read, readv, write, writev, send, recv,
sendmsg, recvmsg have been generalized to read(buf, flags), write(buf,
flags), readVectorized(vectors, flags), and writeVectorized(vectors,
flags). There is still some work left to be done abstracting both
readVectorized and writeVectorized properly across platforms, which is
work to be done in a future PR.
Support for setting the linger timeout of a socket, querying the remote
address of a socket, setting whether or not keep-alive messages are to
be sent through a connection-oriented socket periodically depending on
host operating system settings has been added.
`std.io.Reader` and `std.io.Writer` wrappers around `Socket` has been
implemented, which wrap around Socket.read(buf, flags) and
Socket.write(buf, flags). Both wrappers may be provided flags which are
passed to Socket.read / Socket.write accordingly.
Cross-platform support for `getpeername()` has been implemented.
Windows support for the new `std.x.os.Socket` has been implemented. To
accomplish this, a full refactor of `std.os.windows.ws2_32` has been
done to supply any missing definitions and constants based on
auto-generated Windows syscall bindings by @marler8997.
`std.x.net.TCP.Listener.setQuickACK` has been moved to
`std.x.net.TCP.Client.setQuickACK`.
Windows support for resolving the scope ID of an interface name
specified in an IPv6 address has been provided.
`sockaddr_storage` definitions have been provided for Windows, Linux,
and Darwin. `sockaddr_storage` is used to allocate space before any
socket addresses are queried via. calls such as accept(), getsockname(),
and getpeername().
Zig-friendly wrappers for GetQueuedCompletionStatusEx(), getpeername(),
SetConsoleCtrlHandler(), SetFileCompletionNotificationModes() syscalls
on Windows have been provided.
Socket.setOption() was provided to set the value of a socket option in
place of os.setsockopt. Socket.getOption() will be provided in a future
PR.
There is still further work to be done regarding querying socket option
values on Windows, which is to be done in a subsequent PR.
Instead of multiple references to an anonymous structure to represent affine coordinates, add an actual `AffineCoordinates` structure.
Also properly handle the neutral element during coordinate conversion and fix mixed addition.
And comptime the small precomputation table for basepoint multiplication.
The system `stat` structure includes padding, and, on some
operating systems such as all BSDs, "spare" bytes at the end.
We can't reliably compare two `Stat` values if these are
uninitialized, while being later compared.
This is what was causing the `fstatat` test to fail on FreeBSD
since the update to LLVM 12. It was previously only passing by
accident.
Previously, I mistakenly assumed that offset of the relocation
is enough when calculating relative offset of the target from the
source target section base address in case of section-based relocs
on x86_64. While this is true for `__TEXT,__text` section which
always starts at 0x0 in object files, this is absolutely not true
for `__TEXT,__StaticInit` section which will have nonzero base
address hence resulting in incorrect displacement calculations for
SIGNED relocs.
Conflicts:
* doc/langref.html.in
* lib/std/enums.zig
* lib/std/fmt.zig
* lib/std/hash/auto_hash.zig
* lib/std/math.zig
* lib/std/mem.zig
* lib/std/meta.zig
* test/behavior/alignof.zig
* test/behavior/bitcast.zig
* test/behavior/bugs/1421.zig
* test/behavior/cast.zig
* test/behavior/ptrcast.zig
* test/behavior/type_info.zig
* test/behavior/vector.zig
Master branch added `try` to a bunch of testing function calls, and some
lines also had changed how to refer to the native architecture and other
`@import("builtin")` stuff.
* no more global atomic variable for anonymous decl indexes. Instead
the Namespace decl table is used to figure out anonymous decl names.
- This prepares for better multi-threaded semantic analysis in the
future, with no contention on this global atomic integer.
* implement fully qualified names for namespaces.
Some of the reworkings in this branch put us over the limit, on Linux,
where the kernel disregards the fact that we ask for 16 MiB in the ELF
file. So we ask for more stack space in `main`.
* Sema: implement global variables
- Improved global constants to stop needlessly creating a Var
structure; they can just store the value directly.
- This required making memory management a bit more sophisticated to
detect when a Decl owns the Namespace associated with it, for the
purposes of deinitialization.
* Decl.name and Namespace decl table keys no longer directly
reference ZIR; instead they have heap-duped names, so that deleted
decls, which no longer have any ZIR to reference for their names, can
be removed from the parent Namespace table.
- In the future I would like to explore going a different direction
with this, where the strings would still point to the ZIR however
they would be removed from their owner Namespace objects during the
update detection. The design principle here is that the existence
of incremental compilation as a feature should not incur any cost
for the use case when it is not used. In this example Decl names
could simply point to ZIR string table memory, and it is only
because of incremental compilation that we duplicate their names.
* AstGen: implement threadlocal variables
* CLI: call cleanExit after building a compilation so that in release
modes we don't bother freeing memory or closing file descriptors,
allowing the OS to do it more efficiently.
* Avoid calling `freeDecl` in the linker for unreferenced Decl objects.
* Fix CBE test case expecting the compile error to point to the wrong
column.
After this commit, `pub export fn main() c_int { ... }` will be
correctly detected as the intended entry point, and therefore start code
will not try to export its own conflicting `main` function.
* Implement basic union support
- lots of stuff is still TODO, including runtime field access
- also TODO: resolving the union tag type
- comptime field access is implemented
* DRY up some code by using the `Zir.DeclIterator` for skipping over
decls in structs and unions.
* Start to clean up Sema with regards to calling `.value()` to find out
a const value. Instead, Sema code should call one of these two:
- `resolvePossiblyUndefinedValue` (followed by logic dealing with
undefined values)
- `resolveDefinedValue` (a compile error will be emitted if the value
is undefined)
* An exported function with an unspecified calling convention gets the
C calling convention.
* Implement comptime field access for structs.
* Add another implementation of "type has one possible value" in Sema.
This is a bit unfortunate since the logic is duplicated, but the one
in Type asserts that the types are resolved already, and is
appropriate to call from codegen, while the one in Sema performs
type resolution if necessary, reporting any compile errors that occur
in the process.
The goal is to get start code to be able to inspect the calling
convention of `main` in order to determine whether to export a main for
libc to call, or to allow the root source file to do it.
Coming from other languages it might be tempting for programmers to
accidentally leave out the return type instead of returning 'void'.
The error for this used to be
error: invalid token: '{'
pub fn main() {
^
which is misleading. The '{' is expected but only after a return type.
The new message is
error: expected return type (use 'void' to return nothing), found: '{'
pub fn main() {
^
which not only points out the real error but also hints at a (probably)
very common case where someone coming from e.g. Go is used to not
specifying a return type if a function returns nothing and thus forgets
to put 'void' there.
It might seem overkill to hint at the 'void' option but then the
compiler error messages are our user interface to the programmer. We
can be better than other languages in our error messages and leaving
out the return type seems to be a rather clear indication of the above
mentioned issue. Adding this will help more than distract.
during an incremental update change detection,
the function call to get the old contents hash took place
after mangling the old ZIR index, making it access the wrong
array index.
The results have been cross-checked with LLVM's APFloat implementation
by randomly sampling the f32/f64 space, while the f16 one was completely
checked given the small size.
The presence of a trailing comma in the single and only input/output
declaration confused the parser and made zig fmt discard any element
placed after the comma.
PR #7827 added some new `std.Target.Os.Tag` before `other`.
The corresponding enum in stage1.h was not updated, which caused a
mismatch in the underlying integer values. While attempting to target
`other`, I encountered crashes.
This PR updates the stage1.h enum to include the added OS tags.
The new tags also had to be added to various switch cases to fix
compiler warnings, but have not been tested in any way.
On OpenBSD, connecting to a newly-allocated port on an unspecified IPv4
host address causes EINVAL. This was found by @mikdusan when running TCP
tests on OpenBSD.
This commit fixes TCP tests on OpenBSD by having all tests that allocate
a new host-port pair to have the host IPv4/IPv6 address point to the
host's loopback adapter (on localhost).
There are some small problems here and there, mostly due to the pointers
having the lsb set and disrupting the fn alignment tests and the
`@FrameSize` implementation.
To be honest all this detection logic is starting to become a real PITA,
the ARM32 version can be possibly removed as the generic version
optimizes pretty well...
* UTF16 gets its own section, `__TEXT,__ustring`
* TLV data and bss sections have to aligned to the same max alignment
according to Apple rdar comment in the latest ld64
Now that they are lazy, they need to get analyzed in the correct
context, when requested.
This commit also hooks up std.builtin type values being resolved
properly. This is needed, for example, with the `@export` builtin
function, which occurs in start.zig, for `std.builtin.ExportOptions`.
The ZIR code uses the special `Ref.export_options` value, and semantic
analysis has to map this to the corresponding type from `std.builtin`.
Functions generated by Fiat-crypto are not prefixed by their description any more. This matches an upstream change.
We can now use a single type for different curves and implementations.
The field type is now generic, so we can properly handle the base field and scalars without code duplication.
Use i32 instead of isize for os.timeval's for socket read/write
timeouts.
Add a comptime check to resolveScopeID to see if `IFNAMESIZE` is
available on the host. If it is not available, return an error
indicating that resolving the scope ID of a IPv6 address is not yet
supported on the host platform.
Address comments from @ifreund and @MasterQ32 to address unsafeness and
ergonomics of the `Address` API.
Rename the `TCP` namespace to `tcp` as it does not contain any
top-level fields.
Fix missing reference to `sockaddr` which was identified by @kprotty in
os/bits/linux/arm64.zig.
The `Socket` abstraction was refactored to only comprise of methods that
can be generically used/applied to all socket domains and protocols.
A more comprehensive IPv4/IPv6 module derived from @LemonBoy's earlier
work was implemented under `std.x.os.IPv4` and `std.x.os.IPv6`. Using
this module, one can then combine them together into a union for example
in order to optimize memory usage when dealing with socket addresses.
A `TCP.Client` and `TCP.Listener` abstraction is introduced that is one
layer over the `Socket` abstraction, which isolates methods that can
only be applied to a "client socket" and a "listening socket". All prior
tests from the `Socket` abstraction, which all previously operated
assuming the socket is operating via. TCP/IP, were moved. All TCP socket
options were also moved into the `TCP.Client` and `TCP.Listener`
abstractions respectively away from the `Socket` abstraction.
Some additional socket options from @LemonBoy's prior PR for Darwin were
also moved in (i.e. SIGNOPIPE).
AstGen is now completely independent from the rest of the compiler. It
ingests an AST tree and produces ZIR code as the output, without
depending on any of the glue code of the compiler.
In the byteOffset function, compile errors may need to compute the AST
from source bytes in order to resolve source locations. Previously there
were a few lines trying to access the AST before it was loaded. Trivial
fix, just move load the tree at the beginning.
This allows Sema to namespace them separately from function decls with
the same name. Ran into this in std.math.order conflicting with a test
with the same name.
instead of node indexes.
* AstGen: dbg_stmt instructions now have line and column indexes,
relative to the parent declaration. This allows codegen to emit debug
info without having the source bytes, tokens, or AST nodes loaded
in memory.
* ZIR: each decl has the absolute line number. This allows computing
line numbers from offsets without consulting source code bytes.
Memory management: creating a function definition does not prematurely
set the Decl arena. Instead the function is allocated with the general
purpose allocator.
Codegen no longer looks at source code bytes for any reason. They can
remain unloaded from disk.
Leave the minimum amount of stack space required by the ABI (16 * 8
bytes for the window contents plus 6 * 8 bytes for the input arguments)
on the new thread stack.
Uses verified code generated by fiat-crypto for field arithmetic, and complete formulas to avoid side channels.
There's still plenty of room for optimizations, especially with a fixed base. But this gives us a framework to easily add other similar curves.
Conflicts:
lib/std/crypto/25519/field.zig
lib/std/crypto/poly1305.zig
I had resolved those by removing `comptime` but master branch decided to
make the parameters `comptime`.
This also pulls in the updated default `zig build` install directory.
* AstGen: LocalVal and LocalPtr use string table indexes for their
names. This is more efficient because local variable declarations do
need to include the variable names so that semantic analysis can emit
a compile error if a declaration is shadowed. So we take advantage of
this fact by comparing string table indexes when resolving names.
* The arg ZIR instructions are needed for the above reasoning, as well
as to emit equivalent AIR instructions for debug info.
Now that we have these arg instructions, get rid of the special
`Zir.Inst.Ref` range for parameters. ZIR instructions now refer
to the arg instructions for parameters.
* Move identAsString and strLitAsString from Module.GenZir to AstGen
where they belong.
On CI, we have been running into OOM issues when running the test
suite on Windows for quite some time.
Unfortunately, we are very close to having the same issues on Linux
as well. Some additional comptime work immediately makes these builds
fail as well.
Add a new `test-toolchain` step, that tests everything except `std.*`
and documentation.
On CI, call `test-toolchain`, `test-std` and `docs` separately
instead of the `test` big hammer that emcompasses all of them.
Change the special case we made for Windows to the same code as other
platforms.
This is a stopgap measure that stage2 will eventually make useless.
Until then, it gives us some headroom.
Change `linux_script` by the way to only output the log of failing
steps. This shrinks the Linux CI log from a bazilion lines down to
something more humanely manageable.
stage1 did this by accident by unconditionally semantically analyzing
std.builtin.panic, which imports the root source file to look for a
panic handler override. But stage2 is smarter; it will only semantically
analyze std.builtin.panic if any panic calls are emitted. So for very
simple compilations, the root source file was being ignored, even if it
contained `export` functions in it.
Before there was this "top_decl" and "tmp_namespace" stack values that
were kludgy and buggy. Now Sema is slightly reworked so that files which
are structs are analyzed with their own Decl and Namespace already set
up.
After this commit there are no memory leaks for a successful build-obj.
* Add a yield pattern for PowerPC64
* Fix compile error on pre-v6 ARM targets
* Use isb instead of yield on AArch64 to give the CPU a chance to enter
low-power states.
* Make the hint an inline function, the call overhead can be avoided.
Two problems solved:
* The Decl name may be allocated with gpa or it may be a reference to
the ZIR string table.
* The main update() function was freeing the ZIR when we still had
Decl objects referencing it.
Currently the default install prefix is $BUILD_ROOT/zig-cache,
but mixing cache and artifacts makes little sense. Instead make
$BUILD_ROOT/zig-out the default.
* AstGen: add missing `break_inline` for comptime blocks.
* Module: call getTree() in byteOffset(). This generates the AST when
using cached ZIR and compile errors need to be reported.
* Scope.File: distinguish between successful ZIR generation and AIR
generation (when Decls in scope have been scanned).
- `semaFile` correctly avoids doing work twice.
* Implement first pass at `lookupInNamespace`. It has various TODOs
left, such as `usingnamespace`, and setting up Decl dependencies.
We do this by reserving string table indexes 0 and 1 in ZIR to be
special. Decls now have 0 to mean comptime or usingnamespace, and 1 to
mean an unnamed test decl.
Should be good enough to unblock progress on the stage2 compiler.
Unifying this parser and the regular one (and perhaps rewrite it, #2207)
is left as an exercise for the reader.
* Remove some unused imports in AstGen.zig. I think it would make sense
to start decoupling AstGen from the rest of the compiler code,
similar to how the tokenizer and parser are decoupled.
* AstGen: For decls, move the block_inline instructions to the top of
the function so that they get lower ZIR instruction indexes. With
this, the block_inline instruction index combined with its corresponding
break_inline instruction index can be used to form a ZIR instruction
range. This is useful for allocating an array to map ZIR instructions
to semantically analyzed instructions.
* Module: extract emit-h functionality into a struct, and only allocate
it when emit-h is activated.
* Module: remove the `decl_table` field. This previously was a table of
all Decls in the entire Module. A "name hash" strategy was used to
find decls within a given namespace, using this global table. Now,
each Namespace has its own map of name to children Decls.
- Additionally, there were 3 places that relied on iterating over
decl_table in order to function:
- C backend and SPIR-V backend. These now have their own decl_table
that they keep populated when `updateDecl` and `removeDecl` are
called.
- emit-h. A `decl_table` field has been added to the new GlobalEmitH
struct which is only allocated when emit-h is activated.
* Module: fix ZIR serialization/deserialization bug in debug mode having
to do with the secret safety tag for untagged unions. There is still an
open TODO to investigate a friendlier solution to this problem with
the language.
* Module: improve deserialization of ZIR to allocate only exactly as
much capacity as length in the instructions array so as to not waste
space.
* Module: move `srcHashEql` to `std.zig` to live next to the definition
of `SrcHash` itself.
* Module: re-introduce the logic for scanning top level declarations
within a namespace.
* Compilation: add an `analyze_pkg` Job which is used to kick off the
start of semantic analysis by doing the equivalent of
`_ = @import("std");`. The `analyze_pkg` job is unconditionally added
to the work queue on every update(), with pkg set to the std lib pkg.
* Rename TZIR to AIR in a few places. A more comprehensive rename will
come later.
* Every decl provides a 16 byte source hash which can be used to detect
if the source code for any particular decl has changed.
* Include comptime decls, test decls, and usingnamespace decls in the
decls list of namespaces.
- Tests are encoded as extended functions with is_test bit set.
* 25519: remove unused const, safeguard against unreduced scalars
No behavior change, but it makes the existing code better match
the forthcoming code for other curves.
Rename nonAdjacentForm() to slide(), remove an unneeded and
confusing constant, and do a reduction in slide() if 257 bits would
be required.
Note that in all the high-level functions, the top bit is always
cleared, so the reduction is never necessary. But since the low-level
functions are public, the check is a safe thing to have.
* 25519: make identityElement public, deprecate neutralElement
Also fix a few comments by the way.
This reverts commit 77cb45f59f.
Zig's error return traces will point to the return token if they
happen to occur, so having multiple return statements makes those
stack traces really helpful. This destroys debuggability.
* #8454 Fix for std.mem.replacementSize adjacent matches bug.
When two 'needle' values are adjacent in the 'input' slice, the size is not
counted correctly. The 2nd 'needle' value is not matched because the index is
incremented by one after changing the index to account for the first value.
The impact is the the size returned is incorrect, and could cause UB when this
amount is used to size of the buffer passed to std.mem.replace.
* Apply changes from PR review:
- Add assert checking that the needle is non-empty and doc for this.
- Add minimal test that an empty input works.
- Use testing.expectEqualStrings.
Flush all the register windows to stack before starting the stack walk,
we may otherwise try to read garbage and crash and burn.
Add a few comptime annotations to debloat some functions.
A little function to complement the existing crypto.utils.timingSafeEql
function with a way to compare large numbers serialized as arrays.
This is useful to compare nonces and to check that group elements are
in canonical form.
Absence of side channels remains a best effort, reusing the common
pattern we use elsewhere.
The @ptrCast(X, @alignCast(@alignOf(T), Y)) pattern is only correct if T
is not a function type or a pointer, in that case the @alignOf refers to
the pointer itself and not to the pointee type.
The code would previously assume every function would start at addresses
being multiples of 16, this is not true beside some specific cases.
Moreover LLVM picks different alignment values depending on whether it's
trying to generate dense or fast code.
Let's use the minimum guaranteed alignment as base value, computed
according to how big the opcodes are.
The alignment of function pointers is always 1, a safe value that won't
cause any error at runtime. Note that this was already the case before
this commit, here we're making this choice explicit.
Let the 'alignment' field for TypeInfo of fn types reflect the ABI
alignment used by the compiler, make this field behave similarly to the
'alignment' one for pointers.
Kick-start initial work on new cross-platform abstraction for
sockets. Adds a test for read timeouts and a test for creating
a non-blocking socket pair on Linux.
The new Socket abstraction is barebones and is made to support both
blocking and non-blocking abstractions, alongside different socket
protocols and domains.
Support for platform-dependant socket options that handles unsupported
platforms gracefully via. comptime checks is provided for the new Socket
abstraction.
This also marks the first out of many commits for introducing breaking
changes to the standard library in a separate `x` folder, which was
pre-approved by @andrewrk.
The intent for the new `x` package is to introduce new async, event loop,
networking, and operating system abstractions that would require breaking
the standard library significantly. By having the `x` package, code in the
standard library and compiler may then slowly be refactored to use the `x`
package. Once modules in the `x` package are stabilized, they can be moved
out of the `x` package, and a global 'grep' can be done to update import
paths that resolve to the stabilized module in the `x` package.
Notably this exposed an issue with the language having to do with the
secret safety tag on untagged unions. How can we have our cake and eat
it too? Not solved in this commit. I will file a language proposal to
tackle this issue soon.
Fixes a compile error in `std.fs.File.readvAll`.
Instead of having one function per precedence level, mirroring the BNF grammar,
this uses the "precedence climbing" algorithm with a table of operator
precedences (and a few special cases that don't fit into the table as it is).
This is a first pass -- it's probably possible to put more of the parser
into this form, e.g. to support prefix/suffix operators with precedence, if
necessary, or just to simplify the code more.
It may also be possible to speed this up by putting more useful information
into the tokens during tokenization, to avoid the extra branch on the token in
operInfo.
When trying to retrieve 80bit fp values from clang using
getValueAsApproximateDouble we'd eventually hit the ceiling value and
return infinity, an invalid value for a fp literal.
Add some logic to prevent this error and warn the user.
Closes#8602
Inside a nosuspend block, emit function calls as nosuspend calls.
Also inside a comptime block, emit function calls as comptime calls.
Also emit `async foo()` calls as async calls.
Remove compile error for `nosuspend` block inside `suspend` block.
Instead of implicitly treating every `suspend` block also as a
`nosuspend` block (which would make sense), we leave suspension points
as compile errors, to hint to the programmer about accidents. Of course
they may then assert `nosuspend` by introducing a block within their
suspend block.
To make room in `Zir.Inst.Tag` I moved `typeof_peer` and `compile_log`
to `Extended`.
* AstGen: implement `anyframe_literal` and `anyframe_type`.
* Introduce `makeSubBlock` to avoid redundant AstGen code for GenZir
scopes. Allows adding/removing a field without possibility of
accidentally introducing a bug of forgetting to set the new field.
* Add to GenZir `nosuspend_node` and `suspend_node` in preparation for
implementing `suspend` blocks and `nosuspend` blocks.
* AstGen: fix assembly to support clobbers, multiple outputs, and
outputs without `->` syntax.
- `asm` and `asm_volatile` move to `Extended` enum with `small` being
repurposed for a few things. This frees up 2 ZIR tags, 1 of which
is used in this commit and 1 is leftover.
* AstGen: fix `simple_types` incorrectly having multiple conflicting
values for "undefined" and "null".
- Also add "anyframe" to `simple_types`.
* Add `anyframe_type` to type.zig, value.zig and `Zir.Inst.Ref`.
- Also add i128 and u128 types to `Zir.Inst.Ref` and `simple_types`.
* Sema/Zir: Fix incorrect math causing the function body to be messed
up for Extended-encoded functions.
* Zir: support `i32` fields for "extra" payloads.
* `comptime const` is redundant
* don't use `extern enum`; specify a tag type.
`extern enum` is only when you need tags to alias. But aliasing tags
is a smell. I will be making a proposal shortly to remove `extern enum`
from the language.
* there is no such thing as `packed enum`.
* instead of `catch |_|`, omit the capture entirely.
* unused function definition with missing parameter name
* using `try` outside of a function or test
It turns out that the endianness-detection header delivered with the
softfloat library is extremely brittle and gives wrong results when
targeting FreeBSD (long story short, _BIG_ENDIAN is always defined there
and that breaks the #if defined() chain).
Use our own endianness detection header to work around any potential
problem.
- original PR #7949 (incorrectly) patched a generated-file and changes
have subsequently been lost/overwritten
- fix#7947 in a different way: drop `ppc32` because `ppc` already exists
Primarily this required fixing `setCondBrPayloadElideBlockStorePtr` to
not assume there would always be two `store_to_block_ptr` instructions
in each of the condbr prongs.
std/crypto: use finer-grained error sets in function signatures
Returning the `crypto.Error` error set for all crypto operations
was very convenient to ensure that errors were used consistently,
and to avoid having multiple error names for the same thing.
The flipside is that callers were forced to always handle all
possible errors, even those that could never be returned by a
function.
This PR makes all functions return union sets of the actual errors
they can return.
The error sets themselves are all limited to a single error.
Larger sets are useful for platform-specific APIs, but we don't have
any of these in `std/crypto`, and I couldn't find any meaningful way
to build larger sets.
This commit also reclaims +2 ZIR instruction tags by moving the
following to `extended`:
* func_var_args
* func_extra
* func_extra_var_args
The following ZIR instruction tag is added:
* func_inferred
These are currently incorrect according to the gitattributes(5) and
gitignore(5) man pages. However, it seems github ended up treating them
as we intended due to a bug until recently when that bug was fixed.
I've run into this footgun enough times, nearly every time I want
`ensureUnusedCapacity`, not `ensureCapacity`. This commit deprecates
`ensureCapacity` in favor of `ensureTotalCapacity` and introduces
`ensureUnusedCapacity`.
* AstGen: represent compile errors in ZIR rather than returning
`error.AnalysisFail`.
* ZIR: remove decl_ref and decl_val instructions. These are replaced by
`decl_ref_named` and `decl_val_named`, respectively, which will
probably get renamed in the future to the instructions that were just
deleted.
* AstGen: implement `@This()`, `@fence()`, `@returnAddress()`, and
`@src()`.
* AstGen: struct_decl improved to support fields_len=0 but have decls.
* AstGen: fix missing null bytes after compile error messages.
* SrcLoc: no longer depend on `Decl`. Instead have an explicit field
`parent_decl_node` which is an absolute AST Node index.
* Module: `failed_files` table can have null value, in which case the
key, which is a `*Scope.File`, will have ZIR errors in it.
* ZIR: implement text rendering of struct decls.
* CLI: introduce debug_usage and `zig astgen` command which is enabled
when the compiler is built in debug mode.
This was also an experiment to see if it were easier to implement a new
feature when using the instruction encoder.
Verdict: It's not that much easier, but I think it's certainly much more
readable, because the description of the Instruction annotates what each
field means. Right now, precise knowledge of x86_64 instructions is
still required because things like when to set the 64-bit flag, how to
read x86_64 instruction references, etc. are still not automatically
done for you.
In the future, this interface might make it sligtly easier to write an
assembler for x86_64, by abstracting the bit-fiddling aspects of
instruction encoding.
From my very cursory reading, it seems that the register manager doesn't
distinguish between registers that are physically the same but have
different sizes.
In that case, this means that during codegen, we can't rely on
`reg.size()` when determining the width of the operations we have to
perform. Instead, we must use some form of `ty.abiSize(self.target.*)`
to determine the size of the type we're operating with. If this size is
64 bits, then we should enable 64-bit operation.
This fixed a bug in the codegen for spilling instructions, which was
overwriting the previous stack entry with zeroes. See the modified test
case in this commit.
There are parts of it that I didn't modify because the byte
representation was important (e.g. we need to know what the exact
byte position where we store the address into the offset table is)
In std.meta.cast when casting to an enum type from an integer type, first
do a C-style cast from the source value to the tag type of the enum.
This ensures that we don't get an error due to the source value not being
representable by the enum.
In transCCast() use std.meta.cast instead of directly emitting the cast
operation since the enum's underlying type may not be known at translation
time due to an MSVC bug, see https://github.com/ziglang/zig/issues/8003Fixes#6011
See #8516.
* AstGen is now done on whole files at once rather than per Decl.
* Introduce a new wait group for AstGen tasks. `performAllTheWork`
waits for all AstGen tasks to be complete before doing Sema,
single-threaded.
- The C object compilation tasks are moved to be spawned after
AstGen, since they only need to complete by the end of
the function.
With this commit, the codebase compiles, but much more reworking is
needed to get things back into a useful state.
* AstGen: emit decl lookup ZIR instructions rather than directly
looking up decls in AstGen. This is necessary because we want to
reuse the same immutable ZIR code for multiple generic instantiations
(and comptime function calls).
* AstGen: fix using members_len instead of fields_len for struct decls.
* structs: the struct_decl ZIR instruction is now also a block. This is
so that the type expressions, default field value expressions, and
alignment expressions can be evaluated in a scope that contains the
decls from the struct namespace itself.
* Add "std" and "builtin" packages to the builtin package.
* Don't try to build glibc, musl, or mingw-w64 when using `-ofmt=c`.
* builtin.zig is generated without `usingnamespace`.
* builtin.zig takes advantage of `std.zig.fmtId` for CPU features.
* A first pass at implementing `usingnamespace`. It's problematic and
should either be deleted, or polished, before merging this branch.
* Sema: allow explicitly specifying the namespace in which to look up
Decls. This is used by `struct_decl` in order to put the decls from
the struct namespace itself in scope when evaluating the type
expressions, default value expressions, and alignment expressions.
* Module: fix `analyzeNamespace` assuming that it is the top-level root
declaration node.
* Sema: implement comptime and runtime cmp operator.
* Sema: implement peer type resolution for enums and enum literals.
* Pull in the changes from master branch:
262e09c482.
* ZIR: complete out simple_ptr_type debug printing
Instead of Module setting up the root_scope with the root source file,
instead, Module relies on the package table graph being set up properly,
and inside `update()`, it does the equivalent of `_ = @import("std");`.
This, in term, imports start.zig, which has the logic to call main (or
not). `Module` no longer has `root_scope` - the root source file is no
longer special, it's just in the package table mapped to "root".
I also went ahead and implemented proper detection of updated files.
mtime, inode, size, and source hash are kept in `Scope.File`.
During an update, iterate over `import_table` and stat each file to find
out which ones are updated.
The source hash is redundant with the source hash used by the struct
decl that corresponds to the file, so it should be removed in a future
commit before merging the branch.
* AstGen: add "previously declared here" notes for variables shadowing
decls.
* Parse imports as structs. Module now calls `AstGen.structDeclInner`,
which is called by `AstGen.containerDecl`.
- `importFile` is a bit kludgy with how it handles the top level Decl
that kinda gets merged into the struct decl at the end of the
function. Be on the look out for bugs related to that as well as
possibly cleaner ways to implement this.
* Module: factor out lookupDeclName into lookupIdentifier and lookupNa
* Rename `Scope.Container` to `Scope.Namespace`.
* Delete some dead code.
This branch won't work until `usingnamespace` is implemented because it
relies on `@import("builtin").OutputMode` and `OutputMode` comes from a
`usingnamespace`.
Taking a detour trough a f64 is dangerous as the softfloat library
doesn't like converting sNaN values.
The error went unnoticed as an exception is raised by the library but
the stage1 compiler doesn't give a damn.
Happy LLVM 12 Release Day.
Please note that the llvm 12 tag does not include the latest commit in
the release/12.x branch, which is in fact a bug fix for a regression
that is causing a failure in Zig's test suite.
Zig master branch is tracking release/12.x, and will be enabling the
test that is fixed by that commit.
* Add some more tests for the sqrt/sqrtf implementations.
The idea is to cross-check the software impl with the HW one whenever
possible.
* Fix a broken test, oops.
This allows `break` statements to be directly translated from the original C.
Add a break statement as the last statement of the while loop to ensure we
don't have an infinite loop if no breaks / returns are hit in the switch.
Fixes#8387
Tests a scenario where the linker line has the following:
```
main.o libA.a libB.a
```
where `main.o` pulls a symbol from `libB.a`, which in turn is
dependent on a symbol from `libA.a`.
Store only globals and undefs at the linker level, while all locals
stay scoped to the actual object file they were defined in. This is
fine since the relocations referencing locals will always be resolved
first using the local symbol table before checking for the reference
within the linker's global symbol table.
This also paves the way for proper symbol resolution from within static
and dynamic libraries.
The code initializes twice `t` instead of `t1`, leaving the latter
uninitialized. The problem manifested itself by corrupting the LSBs of
the result in unpredictable ways.
- limit expected-output to main source file;
ie. tolerate changes to start.zig
- when mode != .Debug the function name is now symbolically represented;
ie. tolerate changes in llvm optimizer effects on the callstack
- cleanup how test cases are specified
- add test case predicates for excluding by arch, os or custom fn
Use `PassBuilder::buildO0DefaultPipeline` to build pipeline for -O0
in replacement of `PassBuilder::buildPerModuleDefaultPipeline`. This
affects both normal and LTO settings.
Two redundant Passes - which were added by accident - were also removed
from LTO pipeline.
rowSize used to return null if all the elements were placed on the same
line as the right brace, making the rendering logic skip the whole set
of elements.
Given the usage of rowSize let's just drop the null and always return
the number of elements.
Fixes#8423
This branch adds "builtin" and "std" to the import table when using the
self-hosted backend.
"builtin" gains one additional item:
```
pub const zig_is_stage2 = true; // false when using stage1 backend
```
This allows the std lib to do conditional compilation based on detecting
which backend is being used. This will be removed from builtin as soon
as self-hosted catches up to feature parity with stage1.
Keep a sharp eye out - people are going to be tempted to abuse this.
The general rule of thumb is do not use `builtin.zig_is_stage2`. However
this commit breaks the rule so that we can gain limited start.zig support
as we incrementally improve the self-hosted compiler.
This commit also implements `fullyQualifiedNameHash` and related
functionality, which effectively puts all Decls in their proper
namespaces. `fullyQualifiedName` is not yet implemented.
Stop printing "todo" log messages for test decls unless we are in test
mode.
Add "previous definition here" error notes for Decl name collisions.
This commit does not bring us yet to a newly passing test case.
Here's what I'm working towards:
```zig
const std = @import("std");
export fn main() c_int {
const a = std.fs.base64_alphabet[0];
return a - 'A';
}
```
Current output:
```
$ ./zig-cache/bin/zig build-exe test.zig
test.zig:3:1: error: TODO implement more analyze elemptr
zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty
```
So the next steps are clear:
* Sema: improve elemptr
* AstGen: implement structInitExpr
This refactor inserts an offset table into wasm's data section
where each offset points to the actual data region.
This means we can keep offset indexes consistant and do not
have to perform any computer to determine where in the data section
something like a static string exists. Instead during runtime
it will load the data offset onto the stack.
The following code caused an assertion to be hit:
```
pub fn main() void {
var e: anyerror!c_int = error.Foo;
const i = e catch 69;
assert(69 - i == 0);
}
```
There were several problems, all fixed:
* AstGen was storing field names as references to the original
source code bytes. However, that data would be destroyed when the
source file is updated. Now, it correctly stores the field names in
the Decl arena for the enum. The same fix applies to error set field
names.
* Sema was missing a memset inside `analyzeSwitch`, leaving the "seen
enum fields" array with undefined memory. Now that they are all
properly set to null, the validation works.
* Moved the "enum declared here" note to the end. It looked weird
interrupting the notes for which enum values were missing.
Before, incremental compilation would crash when trying to emit compile
errors for the update after introducing a parse error.
Parse errors are handled by not invalidating any existing semantic
analysis. However, only the parse error must be reported, with all the
other errors suppressed. Once the parse error is fixed, the new file can
be treated as an update to the previously-succeeded update.
* `analyzeContainer` now has an `outdated_decls` set as well as
`deleted_decls`. Instead of queuing up outdated Decls for re-analysis
right away, they are added to this new set. When processing the
`deleted_decls` set, we remove deleted Decls from the
`outdated_decls` set, to avoid deleted Decl pointers from being in
the work_queue. Only after processing the deleted decls do we add
analyze_decl work items to the queue.
* Module.deletion_set is now an `AutoArrayHashMap` rather than `ArrayList`.
`declareDeclDependency` will now remove a Decl from it as appropriate.
When processing the `deletion_set` in `Compilation.performAllTheWork`,
it now assumes all Decl in the set are to be deleted.
* Fix crash when handling parse errors. Currently we unload the
`ast.Tree` if any parse errors occur. Previously the code emitted a
LazySrcLoc pointing to a token index, but then when we try to resolve
the token index to a byte offset to create a compile error message,
the ast.Tree` would be unloaded. Now we use
`LazySrcLoc.byte_abs` instead of `token_abs` so the error message can
be created even with the `ast.Tree` unloaded.
Together, these changes solve a crash that happened with incremental
compilation when Decls were added and removed in some combinations.
off by one error in the bits that describe defaults/field alignments
as soon as we have tests for struct field alignment and default values,
there will be coverage for this.
Now that we're close to supporting all the types, get rid of the `else`
prong and explicitly list out those types that are not yet implemented.
Thanks @g-w1
A simple enum is an enum which has an automatic integer tag type,
all tag values automatically assigned, and no top level declarations.
Such enums are created directly in AstGen and shared by all the
generic/comptime instantiations of the surrounding ZIR code. This
commit implements, but does not yet add any test cases for, simple enums.
A full enum is an enum for which any of the above conditions are not
true. Full enums are created in Sema, and therefore will create a unique
type per generic/comptime instantiation. This commit does not implement
full enums. However the `enum_decl_nonexhaustive` ZIR instruction is
added and the respective Type functions are filled out.
This commit makes an improvement to ZIR code, removing the decls array
and removing the decl_map from AstGen. Instead, decl_ref and
decl_val ZIR instructions index into the `owner_decl.dependencies`
ArrayHashMap. We already need this dependencies array for incremental
compilation purposes, and so repurposing it to also use it for ZIR decl
indexes makes for efficient memory usage.
Similarly, this commit fixes up incorrect memory management by removing
the `const` ZIR instruction. The two places it was used stored memory in
the AstGen arena, which may get freed after Sema. Now it properly sets
up a new anonymous Decl for error sets and uses a normal decl_val
instruction.
The other usage of `const` ZIR instruction was float literals. These are
now changed to use `float` ZIR instruction when the value fits inside
`zir.Inst.Data` and `float128` otherwise.
AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to
have to make some fixes before they will pass tests. Will do that in the
branch before merging.
AstGen: fix struct astgen incorrectly counting decls as fields.
Type/Value: give up on trying to exhaustively list every tag all the
time. This makes the file more manageable. Also found a bug with
i128/u128 this way, since the name of the function was more obvious when
looking at the tag values.
Type: implement abiAlignment and abiSize for structs. This will need to
get more sophisticated at some point, but for now it is progress.
Value: add new `enum_field_index` tag.
Value: add hash_u32, needed when using ArrayHashMap.
As per the other string types, `?[:0]const u8` needs its own case
as otherwise it will raise an error about using `{}` with slices.
There's no reasonable workaround for this, as you would have to
either discount the use of the empty string value or manually
rework the string to be sentinel-terminated at runtime. It's
useful for passing build options to code making use of C libraries
that make strong use of sentinel-terminated arrays for strings.
The avr1 target is a very minimal subset of the AVR ISA, quoting the GCC
manual:
> This ISA is implemented by the minimal AVR core and supported for
> assembler only.
Default to avr2 as GCC and Clang do.
When a connected socket file descriptor on Linux is re-acquired
after being closed, through fuzz testing, it appears that a
subsequent attempt to establish a connection with the file
descriptor causes EALREADY to be reported.
Instead of panicking, choose to return error.ConnectionPending
to allow for users to handle this fairly rare case.
* Switch json testing 'roundTrip()' to use FixedBufferStream, improve error handling, remove comptime from param
* Add 'try' to calls to roundTrip() that can now return an error
* Remove comptime from params in json testing, replace expect(false) with letting error propagate
* Add 'try' to calls to ok() that can now return an error
Co-authored-by: Lewis Gaul <legaul@cisco.com>
Before this change every keypress in the search field causes a browser
history entry, which makes navigating back annoying.
On first keypress in the search field, a new history entry is created.
On subsequent keypresses, the most recent history entry is replaced.
Therefore a typical history after searching and navigating to an entry
might look like
1. documentation root
2. search page "print"
3. docs for `std.debug.print`
Co-authored-by: Žiga Željko <ziga.zeljko@gmail.com>
Output compile errors when signed integer types are used on functions
where the answer might've been a complex number but that functionality hasn't
been implemented.
This applies to sqrt, log, log2, log10 and ln.
A test which used a signed integer was also changed to use an unsigned
integer instead.
Introduce `ResultLoc.none_or_ref` which is used by field access
expressions to avoid unnecessary loads when the field access itself
will do the load. This turns:
```zig
p.y - p.x - p.x
```
from
```zir
%14 = load(%4) node_offset:8:12
%15 = field_val(%14, "y") node_offset:8:13
%16 = load(%4) node_offset:8:18
%17 = field_val(%16, "x") node_offset:8:19
%18 = sub(%15, %17) node_offset:8:16
%19 = load(%4) node_offset:8:24
%20 = field_val(%19, "x") node_offset:8:25
```
to
```zir
%14 = field_val(%4, "y") node_offset:8:13
%15 = field_val(%4, "x") node_offset:8:19
%16 = sub(%14, %15) node_offset:8:16
%17 = field_val(%4, "x") node_offset:8:25
```
Much more compact. This requires `Sema.zirFieldVal` to support both
pointers and non-pointers.
C backend: Implement typedefs for struct types, as well as the following
TZIR instructions:
* mul
* mulwrap
* addwrap
* subwrap
* ref
* struct_field_ptr
Note that add, addwrap, sub, subwrap, mul, mulwrap instructions are all
incorrect currently and need to be updated to properly handle wrapping
and non wrapping for signed and unsigned.
C backend: change indentation delta to 1, to make the output smaller and
to process fewer bytes.
I promise I will add a test case as soon as I fix those warnings that
are being printed for my test case.
- Enable MergeFunctionsPass in non-debug build.
- Verify input and output IR when the assertion is turned on.
- Add AlwaysInlinePass in debug build.
- Add more comments.
* docs: document the nosuspend keyword
* Specify that resuming from suspend is allowed in nosuspend
* Fix the description of the requirements of nosuspend
* Make use of nosuspend in some example code.
This is mainly motivated by the incorrect claim that "there would be
no way to collect the return value of amain, if it were something
other than void".
New ZIR instructions:
* struct_decl_packed
* struct_decl_extern
New TZIR instruction: struct_field_ptr
Introduce `Module.Struct`. It uses `Value` to store default values and
abi alignments.
Implemented Sema.analyzeStructFieldPtr and zirStructDecl.
Some stuff I changed from `@panic("TODO")` to `log.warn("TODO")`.
It's becoming more clear that we need the lazy value mechanism soon;
Type is becoming unruly, and some of these functions have too much logic
given that they don't have any context for memory management or error
reporting.
GenZir struct now has rl_ty_inst field which tracks the result location
type (if any) a block expects all of its results to be coerced to.
Remove a redundant coercion on const local initialization with a
specified type.
Switch expressions, during elision of store_to_block_ptr instructions,
now re-purpose them to be type coercion when the block has a type in the
result location.
* use the proper result location strategy even when there are noreturn
prongs in the switch expression
* when using break_operand strategy, actually omit the
store_to_block_ptr instructions rather than eliding them.
* for both strategies, properly handle noreturn prongs.
The switch_br ZIR instructions are now switch_block instructions. This
avoids a pointless block always surrounding a switchbr in emitted ZIR
code.
Introduce typeof_elem ZIR instruction for getting the type of the
element of a pointer value in 1 instruction.
Change typeof to be un_node, not un_tok.
Introduce switch_capture ZIR instructions for obtaining the capture
value of switch prongs.
Introduce Sema.resolveBody for when you want to extract a *Inst out of a
block and you know that there is only going to be 1 break from it.
What's not working yet: AstGen does not correctly elide
store instructions when it turns out that the result location does not
need to be used as a pointer.
Also Sema validation code for duplicate switch items is not yet
implemented.
The logic for putting ranges into the else prong is moved from AstGen to
Sema. However, logic to emit multi-items the same as single-items cannot
be done until TZIR supports mapping multiple items to the same block of
code. This will be simple to represent when we do the upcoming TZIR memory
layout changes.
Not yet implemented in this commit is the validation of duplicate
values. The trick is going to be emitting error messages with accurate
source locations, without adding extra source nodes to the ZIR
switch instruction.
This will be done by computing the respective AST node based on the
switch node (which we do have available), only when a compile error
occurs and we need to know the source location to attach the message to.
When filling the last (len % 4) bytes of a buffer, the random number n was only being shifted right by 4 bits for each byte instead of 8. A random u16, for example, would always have its middle two nybbles be equal when generated this way. For comparison, Isaac64.zig, Sfc64.zig, and Xoroshiro128.zig all correctly shift right by 8 bits for each of the last bytes in their nearly identical fill functions.
Here's what I think the ZIR should be. AstGen is not yet implemented to
match this, and the main implementation of analyzeSwitch in Sema is not
yet implemented to match it either.
Here are some example byte size reductions from master branch, with the
ZIR memory layout from this commit:
```
switch (foo) {
a => 1,
b => 2,
c => 3,
d => 4,
}
```
184 bytes (master) => 40 bytes (this branch)
```
switch (foo) {
a, b => 1,
c..d, e, f => 2,
g => 3,
else => 4,
}
```
240 bytes (master) => 80 bytes (this branch)
This makes a few changes to the base64 codecs.
* The padding character is optional. The common "URL-safe" variant, in
particular, is generally not used with padding. This is also the case for
password hashes, so having this will avoid code duplication with bcrypt,
scrypt and other functions.
* The URL-safe variant is added. Instead of having individual constants
for each parameter of each variant, we are now grouping these in a
struct. So, `standard_pad_char` just becomes `standard.pad_char`.
* Types are not `snake_case`'d any more. So, `standard_encoder` becomes
`standard.Encoder`, as it is a type.
* Creating a decoder with ignored characters required the alphabet and
padding. Now, `standard.decoderWithIgnore(<ignored chars>)` returns a
decoder with the standard parameters and the set of ignored chars.
* Whatever applies to `standard.*` obviously also works with `url_safe.*`
* the `calcSize()` interface was inconsistent, taking a length in the
encoder, and a slice in the encoder. Rename the variant that takes a
slice to `calcSizeForSlice()`.
* In the decoder with ignored characters, add `calcSizeUpperBound()`,
which is more useful than the one that takes a slice in order to size
a fixed buffer before we have the data.
* Return `error.InvalidCharacter` when the input actually contains
characters that are neither padding nor part of the alphabet. If we
hit a padding issue (which includes extra bits at the end),
consistently return `error.InvalidPadding`.
* Don't keep the `char_in_alphabet` array permanently in a decoder;
it is only required for sanity checks during initialization.
* Tests are unchanged, but now cover both the standard (padded) and
the url-safe (non-padded) variants.
* Add an error set, rename `OutputTooSmallError` to `NoSpaceLeft`
to match the `hex2bin` equivalent.
Also fixed abiAlignment - for pointers it was returning the abi
alignment inside the type, rather than of the pointer itself. There is
now `ptrAlignment` for getting the alignment inside the type of
pointers.
* zir.Code: introduce a decls array. This is so that `decl_val` and
`decl_ref` instructions can refer to a Decl with a u32 and therefore
they can also store a source location. This is needed for proper
compile error reporting.
* astgen uses a hash map to avoid redundantly adding a Decl to the
decls array.
* fixed reporting "instruction illegal outside function body" instead
of the desired message "unable to resolve comptime value".
* astgen skips emitting dbg_stmt instructions in comptime scopes.
* astgen has some logic to avoid adding unnecessary type coercion
instructions for common values.
Introduce "inline" variants of ZIR tags:
* block => block_inline
* repeat => repeat_inline
* break => break_inline
* condbr => condbr_inline
The inline variants perform control flow at compile-time, and they
utilize the return value of `Sema.analyzeBody`.
`analyzeBody` now returns an Index, not a Ref, which is the ZIR index of
a break instruction. This effectively communicates both the intended
break target block as well as the operand, allowing parent blocks to
find out whether they, in turn, should return the break instruction up the
call stack, or accept the operand as the block's result and continue
analyzing instructions in the block.
Additionally:
* removed the deprecated ZIR tag `block_comptime`.
* removed `break_void_node` so that all break instructions use the same Data.
* zir.Code: remove the `root_start` and `root_len` fields. There is now
implied to be a block at index 0 for the root body. This is so that
`break_inline` has something to point at and we no longer need the
special instruction `break_flat`.
* implement source location byteOffset() for .node_offset_if_cond
.node_offset_for_cond is probably redundant and can be deleted.
We don't have `comptime var` supported yet, so this commit adds a test
that at least makes sure the condition is required to be comptime known
for `inline while`.
* Module.addBreak and addBreakVoid return zir.Inst.Index not Ref
because Index is the simpler type and we never need a Ref for these.
* astgen: make noreturn stuff return the unreachable_value and avoid
unnecessary calls to rvalue()
* breakExpr: avoid unnecessary access into the tokens array
* breakExpr: fix incorrect `@intCast` (previously this unsafely
casted an Index to a Ref)
Change some ZIR instructions from un_tok to un_node. Idea here is to
avoid needlessly accessing the tokens array.
Go ahead and finish the catch code in orelseCatchExpr. Otherwise it
would be an easy mistake to get the scopes wrong when updating that
code and introduce a bug.
Delete a function that is now dead code.
* Introduce helper functions on Module.WipZirCode and zir.Code
* Move some logic around
* re-introduce ref_start_index
* prefer usize for local variables + `@intCast` at the end.
Empirically this is easier to optimize.
* Avoid using mem.{bytesAsSlice,sliceAsBytes} because it incurs an
unnecessary multiplication/division which may cause problems for the
optimizer.
* Use a regular enum, not packed, for `Ref`. Memory layout is
guaranteed for enums which specify their tag type. Packed enums have
ABI alignment of 1 byte which is too small.
This provides us greatly increased type safety and prevents the common
mistake of using a zir.Inst.Ref where a zir.Inst.Index was expected or
vice-versa. It also increases the ergonomics of using the typed values
which can be directly referenced with a Ref over the previous zir.Const
approach.
The main pain point is casting between a []Ref and []u32, which could be
alleviated in the future with a new std.mem function.
This is useful for build.zig files to check in some cases, for example
to adhere to the convention of installing config to /etc instead of
/usr/etc on linux when using the /usr prefix. Perhaps std.build will
handle such common cases eventually, but that is not yet the case.
* comment out the failing stage2 test cases
(so that we can uncomment the ones that are newly passing with
further commits)
* Sema: implement negate, negatewrap
* astgen: implement field access, multiline string literals, and
character literals
* Module: when resolving an AST node into a byte offset, use the
main_tokens array, not the firstToken function
* add `Module.setBlockBody` and related functions
* redo astgen for `and` and `or` to use fewer ZIR instructions and
require less processing for comptime known values
* Sema: rework `analyzeBody` function. See the new doc comments in this
commit. Divides ZIR instructions up into 3 categories:
- always noreturn
- never noreturn
- sometimes noreturn
The current plan is to avoid using async and related features in the
stage2 compiler so that we can bootstrap before implementing them.
Having this untested and incomplete code in the codebase increases
friction while working on stage2, in particular when preforming
larger refactors such as the current zir memory layout rework.
Therefore remove all async related code, leaving only error messages
in astgen.
Ensures that if an assignment statement is the sole statement within a
C if statement, for loop, do loop, or do while loop, then when translated
it resides within a block, even though it does not in the original C.
Fixes the following invalid translation:
`if (1) if (1) 2;` -> `if (true) if (true) _ = @as(c_int, 2);`
To this:
```zig
if (true) if (true) {
_ = @as(c_int, 2);
};
```
Fixes#8159
Now that we ship our own linker for MachO by default in both stage1
and stage2, we need a way to enable logs for verbose debugging.
This commit adds `ZIG_ENABLE_LOGGING` cmake option which is equivalent
to stage2's `-Dlog` flag.
To enable it when building stage1 with cmake, add:
```
cmake .. -DZIG_ENABLE_LOGGING=on
```
These were previously implemented as a sub/sub_wrap instruction with a
lhs of 0. Making this separate instructions however allows us to save
some memory as there is no need to store a lhs.
* free Module.Fn ZIR code when destroying the owner Decl
* unreachable_safe and unreachable_unsafe are collapsed into one ZIR
instruction with a safety flag.
* astgen: emit an unreachable instruction for unreachable literals
* don't forget to call deinit on ZIR code
* astgen: implement some builtin functions
The LLVM backend is now structured into 3 different structs, namely
Object, DeclGen and FuncGen. Object represents an object that is
generated by the LLVM backend. DeclGen is responsible for generating
a decl and FuncGen is responsible for generating llvm instructions
from tzir in a function.
new pipeline `BuildMacOS_arm64`
- `vmImage: 'macOS-10.15' `
new `macos_arm64_script`
- switch from using `make` to `ninja`
- select xcode 12.4
- set zig-cache env variables
- build host-zig binary with xcode, link against llvm for x86_64 (target macos 10.15)
- build arm64-zig binary with xcode and host-zig, link against llvm for arm64 (target macos 11.0)
- ad-hoc codesign arm64 binary with linker
- use host-zig for docgen
- use host-zig for experimental std lib docs
- sync final `release/` hierarchy with `linux_script`
- use gnu-tar for good-practices (set owner, set sort)
enhance `CMakeLists.txt`
- do not build `zig0` when cross-compiling
- disable `BYPRODUCTS` directive `zig1.o` to avoid `ninja` error
see #8265
We are now passing this test:
```zig
export fn _start() noreturn {}
```
```
test.zig:1:30: error: expected noreturn, found void
```
I ran into an issue where we get an integer overflow trying to compute
node index offsets from the containing Decl. The problem is that the
parser adds the Decl node after adding the child nodes. For some things,
it is easy to reserve the node index and then set it later, however, for
this case, it is not a trivial code change, because depending on tokens
after parsing the decl determines whether we want to add a new node or
not.
Possible strategies here:
1. Rework the parser code to make sure that Decl nodes are before
children nodes in the AST node array.
2. Use signed integers for Decl node offsets.
3. Just flip the order of subtraction and addition. Expect Decl Node
index to be greater than children Node indexes.
I opted for (3) because it seems like the simplest thing to do. We'll
want to unify the logic for computing the offsets though because if the
logic gets repeated, it will probably get repeated wrong.
There are some `@panic("TODO")` in there but I'm trying to get the
branch to the point where collaborators can jump in.
Next is to repair the seam between LazySrcLoc and codegen's expected
absolute file offsets.
move a boolean field to be represented implicitly with the enum tag.
Just borrowing one of the many strategies of stage2.
This simple change took the peak mem usage from std lib tests on
my machine from 8.21 GiB to 8.11 GiB.
* translate-c: Use [N:0] arrays when initializer is a string literal
Translate incomplete arrays as [N:0] when initialized by a string literal.
This preserves a bit more of the type information from the original C program.
Fixes#8215
Next up is reworking the seam between the LazySrcLoc emitted by Sema
and the byte offsets currently expected by codegen.
And then the big one: updating astgen.zig to use the new memory layout.
See https://eprint.iacr.org/2019/1492.pdf for justification.
8 rounds ChaCha20 provides a 2.5x speedup, and is still believed
to be safe.
Round-reduced versions are actually deployed (ex: Android filesystem
encryption), and thanks to the magic of comptime, it doesn't take much
to support them.
This also makes the ChaCha20 code more consistent with the Salsa20 code,
removing internal functions that were not part of the public API any more.
No breaking changes; the public API remains backwards compatible.
Now reading a source file and decoding it from UTF-16LE to UTF-8 is
done in a single function. Error messages are improved, and an error is
emitted when the source file has a BOM not supported (UTF-16BE, UTF-32).
Please note that the BOM of UTF-32 is composed of the same bytes as the
BOM of UTF-16 followed by a null character. Therefore a source file in
UTF-16LE starting with a null byte will be interpreted as an UTF-32, and
rejeted because of an invalid format. In pratice this is not a problem,
as the code would have been rejected later anyway because of the null
character.
Fixes#2820
After reading the source code, the first two bytes are inspected, and
if they correspond to a UTF-16 BOM in little-endian order, the source
code is converted to UTF-8.
Bring this in line with how variable declarations are handled.
Open a new indentation level for the initialization expression to handle
nested expressions like blocks.
Closes#7618
The memory layout for ZIR instructions is completely reworked. See
zir.zig for those changes. Some new types:
* `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating
each instruction independently, there is now a Tag and 8 bytes of
data available for all ZIR instructions. Small instructions fit
within these 8 bytes; larger ones use 4 bytes for an index into
`extra`. There is also `string_bytes` so that we can have 4 byte
references to strings. `zir.Inst.Tag` describes how to interpret
those 8 bytes of data.
- This is shared by all `Block` scopes.
* `Module.WipZirCode`: represents an in-progress `zir.Code`. In this
structure, the arrays are mutable, and get resized as we add/delete
things. There is extra state to keep track of things. This struct is
stored on the stack. Once it is finished, it produces an immutable
`zir.Code`, which will remain on the heap for the duration of a
function's existence.
- This is shared by all `GenZir` scopes.
* `Sema`: represents in-progress semantic analysis of a `zir.Code`.
This data is stored on the stack and is shared among all `Block`
scopes. It is now the main "self" argument to everything in the file
that was previously named `zir_sema.zig`.
Additionally, I moved some logic that was in `Module` into here.
`Module.Fn` now stores its parameter names inside the `zir.Code`,
instead of inside ZIR instructions. When the TZIR memory layout
reworking time comes, codegen will be able to reference this data
directly instead of duplicating it.
astgen.zig is (so far) almost entirely untouched, but nearly all of it
will need to be reworked to adhere to this new memory layout structure.
I have no benchmarks to report yet, as I am still working through
compile errors and fixing various things that I broke in this branch.
Overhaul of Source Locations:
Previously we used `usize` everywhere to mean byte offset, but sometimes
also mean other stuff. This was error prone and also made us do
unnecessary work, and store unnecessary bytes in memory.
Now there are more types involved into source locations, and more ways
to describe a source location.
* AllErrors.Message: embrace the assumption that files always have less
than 2 << 32 bytes.
* SrcLoc gets more complicated, to model more complicated source
locations.
* Introduce LazySrcLoc, which can model interesting source locations
with very little stored state. Useful for avoiding doing unnecessary
work when no compile errors occur.
Also, previously, we had `src: usize` on every ZIR instruction. This is
no longer the case. Each instruction now determines whether it even cares
about source location, and if so, how that source location is stored.
This requires more careful work inside `Sema`, but it results in fewer
bytes stored on the heap, without compromising accuracy and power of
compile error messages.
Miscellaneous:
* std.zig: string literals have more helpful result values for
reporting errors. There is now a lower level API and a higher level
API.
- side note: I noticed that the string literal logic needs some love.
There is some unnecessarily hacky code there.
* cut & pasted some TZIR logic that was in zir.zig to ir.zig. This
probably broke stuff and needs to get fixed.
* Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't
think this quite how this code will be organized. Need some more
careful planning about how to implement structs, unions, enums. They
need to be independent Decls, just like a top level function.
The main realization here was that getting rid of the early returns
in renderWhile() and rewriting the logic into a mostly unified execution
path took things from ~200 lines to ~100 lines and improved consistency
by deduplicating code.
Also add several test cases and fix a few issues along the way:
Fixes https://github.com/ziglang/zig/issues/6114
Fixes https://github.com/ziglang/zig/issues/8022
Add failing testcase to reproduce issue 8088
Tidy up renderWhile(), factoring out renderWhilePayload()
Ensure correct newline is used before 'then' token in while/for/if
Handle indents for 'if' inside 'for' or 'while'
Stop special-casing 'if' compared to 'for' and 'while'
fix drivers: zig {cc,c++}
- `-dynamiclib` is an alias to `-shared`
- associate `-dynamiclib` with zig `.shared` semantics
fix drivers: zig {cc,c++,build-lib,build-exe}
- use `-dynamic` for {exe,dylib}
- for dylib this fixes a regression
- for exe this replaces incorrect use of `-static`
A warning is emitted when using the debug option --debug-log when the compiler
was not compiled using the build option -Dlog. Additionnaly, the scopes are not
added to log_scopes as they have no effect.
Previously, there was an option ZIG_PREFER_LLVM_CONFIG which would
override the default of not using llvm-config when cross compiling.
That option is now removed in favor of the more powerful
ZIG_USE_LLVM_CONFIG which defaults to OFF for cross compiling and ON for
native compilation. The option overrides the default.
This will be used in zig-bootstrap to improve support for native builds.
liburing commit: 1bafb3ce5f
As stated in the liburing commit message, this fixes a regression,
reverting code that was added specutively to avoid a syscall in some
cases.
The current implementation of the target C ABI rules is hopelessly bad,
let's tack some more rules on top in order to prevent some
miscompilations.
Truth to be told the same rule should be applied also to parameters, but
I really can't stand stage1.
The modification to the grammar in the comment is in line with the
grammar in the zig-spec repo.
Note: checking if the previous token is a colon is insufficent to tell
if a block has a label, the identifier must be checked for as well. This
can be seen in sentinel terminated slicing: `foo[0..1:{}]`
In order to update the printed progress string the code tried to move
the cursor N cells to the left, where N is the number of written bytes,
and then clear the remaining part of the line.
This strategy has two main issues:
- Is only valid if the number of characters is equal to the number of
written bytes,
- Is only valid if the line doesn't get too long.
The second point is the main motivation for this change, when the line
becomes too long the terminal wraps it to a new physical line. This
means that moving the cursor to the left won't be enough anymore as once
the left border is reached it cannot move anymore.
The wrapped line is still stored by the terminal as a single line,
despite now taking more than a single one when displayed. If you try to
resize the terminal you'll notice how the contents are reflowed and are
essentially illegible.
Querying the cursor position on non-Windows systems (plot twist,
Microsoft suggests using VT escape sequences on newer systems) is
extremely cumbersome so let's do something different.
Before printing anything let's save the cursor position and clear the
screen below the cursor, this way we ensure there's absolutely no trace
of stale data on screen, and after the message is printed we simply
restore it.
Given a pointer operand `ptr` and a signed integer operand `idx`
`ptr + idx` and `idx + ptr` -> ptr + @bitCast(usize, @intCast(isize, idx))
`ptr - idx` -> ptr - @bitCast(usize, @intCast(isize, idx))
Thanks @LemonBoy for pointing out that we can take advantage of wraparound
to dramatically simplify the code.
Clang docs say:
> Like -MMD, but also implies -E and writes to stdout by default.
Previously, Zig handled this option by forwarding it directly to Clang,
and disabling depfiles. However this did not adhere to Clang's documented
behavior of these flags.
Now, in addition to being forwarded directly to Clang, `-MM` also
sets c_out_mode = .preprocessor, just like `-E`.
Another issue I noticed is that Zig did not recognize the aliases for
-MG, -MM, or -MMD. The aliases are now recognized.
The presence of ZIG_VERBOSE_LINK now enables --verbose-link.
The presence of ZIG_VERBOSE_CC now enables --verbose-cc.
These are useful when debugging usage of `zig cc` which does not have
CLI flags for these options, since they are not valid C compiler flags.
* Now it supports being an lvalue (see additional lines in the test
case).
* Properly handles a pointer result location (see additional lines in
the test case that assign the result of the orelse to a variable
rather than a const).
* Properly sets the result location type when possible, so that type
inference of an `orelse` operand expression knows its result type.
We can now codegen optionals! This includes the following instructions:
- is_null
- is_null_ptr
- is_non_null
- is_non_null_ptr
- optional_payload
- optional_payload_ptr
- br_void
Also includes a test for optionals.
Currently `// zig fmt: off` does not work as there are two spaces
after the `//` instead of one. This can cause confusion, so allow
arbitrary whitespace before the `zig fmt: (off|on)` in the comment but
trim this whitespace to the canonical single space in the output.
Let's follow the road paved by the removal of 'z'/'Z', the Formatter
pattern is nice enough to let us remove the remaining four special cases
and declare u8 slices free from any special casing!
LLVM 12 included a patch that changed the way availability annotations
are specified. We now have to define the _LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS
flag to make sure that we tell the c++ headers that we don't use
visibility annotations.
Related LLVM patch: D90843
OCB has been around for a long time.
It's simpler, faster and more secure than AES-GCM.
RFC 7253 was published in 2014. OCB also won the CAESAR competition
along with AEGIS.
It's been implemented in OpenSSL and other libraries for years.
So, why isn't everybody using it instead of GCM? And why don't we
have it in Zig already?
The sad reason for this was patents. GCM was invented only to work
around these patents, and for all this time, OCB was that nice
thing that everybody knew existed but that couldn't be freely used.
That just changed. The OCB patents are now abandoned, and OCB's
author just announced that OCB was officially public domain.
Add support for OffsetOfExpr that contain exactly 1 component, when that component
is a field.
For example, given:
```c
struct S {
float f;
double d;
};
struct T {
long l;
int i;
struct S s[10];
};
```
Then:
```c
offsetof(struct T, i) // supported
offsetof(struct T, s[2].d) // not supported currently
```
Beside the new order being consistent with the ThreadPool API and making
more sense, this shuffling allows to write the context argument type in
terms of the startFn arguments, reducing the use of anytype (eg. less
explicit casts when using comptime_int parameters, yay).
Sorry for the breakage.
Closes#8082
Our glibc stub assembly file looked something like this:
```
.globl _Exit_2_2_5
.type _Exit_2_2_5, %function;
.symver _Exit_2_2_5, _Exit@@GLIBC_2.2.5
.hidden _Exit_2_2_5
_Exit_2_2_5:
```
With clang 12, the shared objects this produced stopped having any
exported symbols. When I removed the `.hidden` directive, it resolved
the issue, however, there are now unwanted exports:
```
$ readelf -W --dyn-syms libc.so.6 | grep sys_errlist
139: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist_GLIBC_2_3
147: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist_GLIBC_2_4
395: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist_GLIBC_2_2_5
487: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist_GLIBC_2_2_5
1266: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@@GLIBC_2.12
1267: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@GLIBC_2.2.5
1268: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@GLIBC_2.3
1269: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@GLIBC_2.4
2137: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@@GLIBC_2.12
2138: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@GLIBC_2.2.5
2139: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@GLIBC_2.3
2140: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@GLIBC_2.4
2156: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist_GLIBC_2_3
2161: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist_GLIBC_2_4
```
Every line here without an `@` symbol is an unwanted export. Before, the
unwanted ones had LOCAL HIDDEN linkage.
As a mitigation, I did two things:
* Added `_GLIBC_` to the unwanted exports so that they would not
conflict with anything.
* Made the default export (the `@@` one) the bare symbol name. This
appears to reduce the unwanted exports to only symbols that have more
than one symbol (which is still quite many).
This will unblock progress on this branch, however, there is now a new
issue to solve, that the provided glibc stub .so files have too many
symbols exported. We will have to find a way to avoid this.
This completes the process. All target CPU features are now
auto-generated by the tools/update_cpu_features.zig script, which
contains all the overrides.
Invoking this tool against LLVM 12rc2 now produces an empty git diff.
With this change, added & modified cpus & features participate in the
same pruning system, and sorting takes into account the zig name, not
the pre-modified llvm name.
The modified target files in this commit are due to the improved
sorting and pruning.
The script now fully supports extra cpus & features.
The tools/update_cpu_features script is coming along, and generates
correct information for all these targets. The remaining targets are:
* arm
* aarch64
* amdgpu
* riscv
I will commit them once the issues with the updater tool are resolved.
This introduces {'} to indicate escape for a single-quoted string,
and {} to indicate escape for a double quoted string.
Without this, there would be unnecessary \' inside double quoted
strings, and unnecessary \" inside single quoted strings.
Motivated by the llvm12 branch, in the new tool I am writing for
updating target CPU features.
also avoid unnecessary escaping of single quotes inside double quoted
strings (depends on a master branch commit that will be merged into this
branch in a future commit)
This replaces the previous target cpu features tool, taking advantage of
llvm-tblgen --dump-json instead of trying to use python to parse the .td
files.
This is an initial version that has the basics working, including a
simple feature override system, as well as multi-threaded processing.
Follow-up commits will do clean ups to make the diff of the newly generated
source files against previous versions be as desired.
LLVM 12 requires sret attributes to have the struct type as a parameter,
and provides no C function for supplying it. Therefore, we must add
another C++ wrapper API for adding the sret attribute.
Fixes ability to build from source in the llvm12 branch.
Conflicts:
* src/clang.zig
* src/llvm.zig
- this file got moved to src/llvm/bindings.zig in master branch so I
had to put the new LLVM arch/os enum tags into it.
* lib/std/target.zig, src/stage1/target.cpp
- haiku had an inconsistency with its default target ABI, gnu vs
eabi. In this commit we make it gnu in both places to match the
latest changes by @hoanga.
* src/translate_c.zig
This is an accident from a merge conflict introduced in
7edb204edf.
The new pipe2 code I believe is supposed to work for all posix-like
systems. If haiku needs special handling here, it should be
re-introduced.
* no isHaiku() function since there is not more than one os tag that
this applies to.
* clean up some control flow into a switch
* add some TODO comments to investigate panics that suspiciously look
like they should be compile errors (see #363)
Add a new allocated_registers bitmap to keep track of all callee-saved
registers allocated during generation of this function.
Function(.arm).gen uses this data to generate instructions in the
function prologue and epilogue to push and pop these registers
respectively.
When two pointers are subtracted, both shall point to elements of the
same array object, or one past the last element of the array object;
the result is the difference of the subscripts of the two array elements.
The size of the result is implementation-defined, and its type
(a signed integer type) is ptrdiff_t defined in the <stddef.h> header.
If the result is not representable in an object of that type,
the behavior is undefined.
See C Standard, §6.5.6 [ISO/IEC 9899:2011]
Fixes#7216
Reverts bf642204b3 and uses a different
workaround, suggested by @LemonBoy.
There is either a compiler bug or a design flaw somewhere around here.
It does not have to block this branch, but I need to understand exactly
what's going on here and make it so that nobody ever has to run into
this problem again.
This regresses the test case of `zig fmt` deleting empty line comments.
Two open questions here:
* What should the rules be about deleting empty line comments?
It makes sense usually, but for array initization, empty line
comments cause a line break, affecting the row/column alignment.
Perhaps we should therefore respect all empty line comments?
Or should we special case array initializations?
* If we decide to special case some kinds of line comments to respect
them (which is status quo!), how should that be implemented?
I modified this test case to expect different results.
Now, the trailing comma on a list of struct fields is the only deciding
factor, not whether or not the field init expressions contain a newline.
Achieve this by reducing the amount of special casing to handle EOF so
that the already correct logic for normal comments does not need to be
duplicated.
After #35 is implemented,
we should be able to recover from this *at any indentation level*,
reporting a parse error and yet also parsing all the decls even
inside structs. Until then, I don't want to add any hacks to make
this work.
I don't understand the idea here of this kind of recovery. If we
want to resurrect this test case we need some comments on it to explain
the purpose, example use cases, expected behavior, etc.
* Added support for passing write file args as build options
* Fix missing fmtEscapes and unused format
* Actually fixed now, must be formatted
* remove addPathBuildOption
The z/Z format specifiers were merged last October (4 months ago). They were then deprecated in January (just over a month ago). This PR removes them altogether.
If the type is a reference to a global declaration that has not yet
been translated we need to use the global scope for translation
so that other functions can also reference it.
All stage2 tests are passing again in this branch.
Remaining checklist for this branch:
* get the rest of the zig fmt test cases passing
- re-enable the translate-c test case that is blocking on this
* implement the 2 `@panic(TODO)`'s in parse.zig
* use fn_proto not fn_decl for extern function declarations
Allowing same line doc comments causes some ambiguity as to how
generated docs should represent the case in which both same line
and preceding line doc comments are present:
/// preceding line
const foobar = 42; /// same line
Furthermore disallowing these makes things simpler as there is now only
one way to add a doc comment to a decl or struct field.
Now it builds and what remains in this branch is:
* fix the stage2 compiler regressions from this branch
* finish the rest of zig fmt test cases, get them passing
* Merge in Vexu's translate-c AST branch & fix translate-c regressions
The previous iteration of translate-c used an incorrect block label
in the break statement for a translated C statement expression. This adds
a test to ensure the correct label is used in the new intermediate AST
version of translate-c.
Starting an async function call is actually a synchronous operation,
since the caller is not awaiting on the callee for a return value.
This commit removes the compiler code which generates the error and
updates the relevant test case.
In the presence of CallModifierAsync, the callee is expected to
suspend, so it should not be changed to CallModifierNoSuspend.
Resuming a suspended async function call is actually a synchronous
operation.
This commit removes the compiler code which generates the error and
updates the relevant test case.
Using this in its current state would be a bug as it could cause line
comments to be deleted or a `// zig fmt: (on|off)` directive to be
missed.
Removing it doesn't currently cause any test failures, if a reason for
its continued existence is discovered in the future another solution
will have to be found.
This type is not widely applicable enough to be a public part of the
public interface of the std.
The current implementation in only fully utilized by the zig fmt
implementation, which could benefit by even tighter integration as
will be demonstrated in the next commit. Therefore, move the current
io.AutoIndentingStream to lib/std/zig/render.zig.
The C backend of the self hosted compiler also use this type currently,
but it does not require anywhere near its full complexity. Therefore,
implement a greatly simplified version of this interface in
src/codegen/c.zig.
I noticed that the write function does not properly use non-blocking
I/O. This file needs to be reworked for evented I/O to properly take
advantage of non-blocking writes to network sockets.
additionally introduce a new file to centralize all the data about
builtin functions that we have, including:
* enum tag identifying the builtin function
* number of parameters.
* whether the expression may need a memory location.
* whether the expression allows an lvalue (currently only true for
`@field`).
Now there is only one ComptimeStringMap that has this data as the value,
and we dispatch on the enum tag in order to asgen the builtin function.
In particular this simplifies the logic for checking the number of
parameters.
This removes some untested code paths from if and while, which need to
be restored with #7929 in mind.
After this there are only a handful left of expression types to rework
to the new memory layout, and then it will be only compile errors left
to solve.
break, continue, blocks, bit_not, negation, identifiers, string
literals, integer literals, inline assembly
also gave multiline string literals a different node tag from regular
string literals, for code clarity and to avoid an unnecessary load from
token_tags array.
Conflicts:
* lib/std/zig/ast.zig
* lib/std/zig/parse.zig
* lib/std/zig/parser_test.zig
* lib/std/zig/render.zig
* src/Module.zig
* src/zir.zig
I resolved some of the conflicts by reverting a small portion of
@tadeokondrak's stage2 logic here regarding `callconv(.Inline)`.
It will need to get reworked as part of this branch.
This commit does not reach any particular milestone, it is
work-in-progress towards getting things to build.
There's a `@panic("TODO")` in translate-c that should be removed when
working on translate-c stuff.
I've added more of the ".def" files from mingw. The list is based on all the libraries referenced by the win32metadata project. (see https://github.com/marlersoft/zigwin32).
In C, if a function has return type `int` and the return expression
is a boolean expression, there is no implicit cast. Therefore the
translated Zig code needs to call @boolToInt() on the result.
Written with feedback from @Vexu
Fixes#6215
Previously, this would reuse an operand even if reuseOperand returned
false for both operands.
genArmBinOpCode was also changed to be more Three-address code oriented
in the process.
Previously, the registers included r0, r1, r2, r3 which are not
included in the callee saved registers according to the Procedure Call
Standard for the ARM Architecture.
The fact that blocks may end in a semicolon but this semicolon is not
counted by recursive lastToken() evaluation on the sub expression causes
off-by-one errors for lastToken() on blocks currently.
To fix this, introduce BlockSemicolon and BlockTwoSemicolon following
the pattern used for trailing commas in e.g. builtin function arguments.
rename PtrType => PtrTypeBitRange, SliceType => PtrType
This rename was done as the current SliceType is used for non-bitrange
pointers as well as slices and because PtrTypeSentinel/PtrTypeAligned
are also used for slices. Therefore using the same Ptr prefix for all
these pointer/slice nodes is an improvement.
Explicitly set the alignment requirements to 1 (i.e, mark the load as unaligned)
since there are some architectures (e.g SPARCv9) which has different alignment
requirements between a function pointer and usize pointer. On those
architectures, not explicitly setting it will lead into @frameSize generating
usize-aligned load instruction that could crash if the function pointer happens
to be not usize-aligned.
* thread/condition: fix PthreadCondition compilation
* thread/condition: add wait, signal and broadcast
This is like std.Thread.Mutex which forwards calls to `impl`; avoids
having to call `cond.impl` every time.
* thread/condition: initialize the implementation
After a right shift, top limbs may be all zero. However, without
normalization, the number of limbs is not going to change.
In order to check if a big number is zero, we used to assume that the
number of limbs is 1. Which may not be the case after right shifts,
even if the actual value is zero.
- Normalize after a right shift
- Add a test for that issue
- Check all the limbs in `eqlZero()`. It may not be necessary if
callers always remember to normalize before calling the function.
But checking all the limbs is very cheap and makes the function less
bug-prone.
The astgen for switch expressions did not respect the ZIR rules of only
referencing instructions that are in scope:
%14 = block_comptime_flat({
%15 = block_comptime_flat({
%16 = const(TypedValue{ .ty = comptime_int, .val = 1})
})
%17 = block_comptime_flat({
%18 = const(TypedValue{ .ty = comptime_int, .val = 2})
})
})
%19 = block({
%20 = ref(%5)
%21 = deref(%20)
%22 = switchbr(%20, [%15, %17], {
%15 => {
%23 = const(TypedValue{ .ty = comptime_int, .val = 1})
%24 = store(%10, %23)
%25 = const(TypedValue{ .ty = void, .val = {}})
%26 = break("label_19", %25)
},
%17 => {
%27 = const(TypedValue{ .ty = comptime_int, .val = 2})
%28 = store(%10, %27)
%29 = const(TypedValue{ .ty = void, .val = {}})
%30 = break("label_19", %29)
}
}, {
%31 = unreachable_safe()
}, special_prong=else)
})
In this snippet you can see that the comptime expr referenced %15 and
%17 which are not in scope. There also was no test coverage for runtime
switch expressions.
Switch expressions will have to be re-introduced to follow these rules
and with some test coverage. There is some usable code being deleted in
this commit; it will be useful to reference when re-implementing switch
later.
A few more improvements to do while we're at it:
* only use .ref result loc on switch target if any prongs obtain the
payload with |*syntax|
- this improvement should be done to if, while, and for as well.
- this will remove the needless ref/deref instructions above
* remove switchbr and add switch_block, which is both a block and a
switch branch.
- similarly we should remove loop and add loop_block.
This commit introduces a "force_comptime" flag into the GenZIR
scope. The main purpose of this will be to choose the "comptime"
variants of certain key zir instructions, such as function calls and
branches. We will be moving away from using the block_comptime_flat
ZIR instruction, and eventually deleting it.
This commit also contains miscellaneous fixes to this branch that bring
it to the state of passing all the tests.
on the break instruction operands. This involves a new TZIR instruction,
br_block_flat, which represents a break instruction where the operand is
the result of a flat block. See the doc comments on the instructions for
more details.
How it works: when adding break instructions in semantic analysis, the
underlying allocation is slightly padded so that it is the size of a
br_block_flat instruction, which allows the break instruction to later
be converted without removing instructions inside the parent body. The
extra type coercion instructions go into the body of the br_block_flat,
and backends are responsible for dispatching the instruction correctly
(it should map to the same function calls for related instructions).
Local variable declarations now detect whether the result location for the
initialization expression consumes the result location as a pointer. If
it does, then the local is emitted as a LocalPtr. Otherwise it is
emitted as a LocalVal.
This results in clean, straightforward ZIR code for semantic analysis.
Motivating test case:
```zig
export fn _start() noreturn {
var x: u64 = 1;
var y: u32 = 2;
var thing: u32 = 1;
const result = if (thing == 1) x else y;
exit();
}
```
The main idea here is for astgen to output ideal ZIR depending on
whether or not the sub-expressions of a block consume the result
location. Here, neither `x` nor `y` consume the result location of the
conditional expression block, and so the ZIR should communicate the
result of the condbr using break instructions, not with the result
location pointer.
With this commit, this is accomplished:
```
%22 = alloc_inferred()
%23 = block({
%24 = const(TypedValue{ .ty = type, .val = bool})
%25 = deref(%18)
%26 = const(TypedValue{ .ty = comptime_int, .val = 1})
%27 = cmp_eq(%25, %26)
%28 = as(%24, %27)
%29 = condbr(%28, {
%30 = deref(%4)
< there is no longer a store instruction here >
%31 = break("label_23", %30)
}, {
%32 = deref(%11)
< there is no longer a store instruction here >
%33 = break("label_23", %32)
})
})
%34 = store_to_inferred_ptr(%22, %23) <-- the store is only here
%35 = resolve_inferred_alloc(%22)
```
However if the result location gets consumed, the break instructions
change to break_void, and the result value is communicated only by the
stores, not by the break instructions.
Implementation:
* The GenZIR scope that conditional branches uses now has an optional
result location pointer field and a count of how many times the
result location ended up being an rvalue (not consumed).
* When rvalue() is called on a result location for a block, it
increments this counter. After generating the branches of a block,
astgen for the conditional branch checks this count and if it is 2
then the store_to_block_ptr instructions are elided and it calls
rvalue() using the block result (which will account for peer type
resolution on the break operands).
astgen has many functions disabled until they can be reworked with these
new semantics. That will be done before merging the branch.
There are some new rules for astgen to follow regarding result locations
and what you are allowed/required to do depending on which one is passed
to expr(). See the updated doc comments of ResultLoc for details.
I also changed naming conventions of stuff in this commit, sorry about
that.
This is a proof-of-concept of switching to a new memory layout for
tokens and AST nodes. The goal is threefold:
* smaller memory footprint
* faster performance for tokenization and parsing
* most importantly, a proof-of-concept that can be also applied to ZIR
and TZIR to improve the entire compiler pipeline in this way.
I had a few key insights here:
* Underlying premise: using less memory will make things faster, because
of fewer allocations and better cache utilization. Also using less
memory is valuable in and of itself.
* Using a Struct-Of-Arrays for tokens and AST nodes, saves the bytes of
padding between the enum tag (which kind of token is it; which kind
of AST node is it) and the next fields in the struct. It also improves
cache coherence, since one can peek ahead in the tokens array without
having to load the source locations of tokens.
* Token memory can be conserved by only having the tag (1 byte) and byte
offset (4 bytes) for a total of 5 bytes per token. It is not necessary
to store the token ending byte offset because one can always re-tokenize
later, but also most tokens the length can be trivially determined from
the tag alone, and for ones where it doesn't, string literals for
example, one must parse the string literal again later anyway in
astgen, making it free to re-tokenize.
* AST nodes do not actually need to store more than 1 token index because
one can poke left and right in the tokens array very cheaply.
So far we are left with one big problem though: how can we put AST nodes
into an array, since different AST nodes are different sizes?
This is where my key observation comes in: one can have a hash table for
the extra data for the less common AST nodes! But it gets even better than
that:
I defined this data that is always present for every AST Node:
* tag (1 byte)
- which AST node is it
* main_token (4 bytes, index into tokens array)
- the tag determines which token this points to
* struct{lhs: u32, rhs: u32}
- enough to store 2 indexes to other AST nodes, the tag determines
how to interpret this data
You can see how a binary operation, such as `a * b` would fit into this
structure perfectly. A unary operation, such as `*a` would also fit,
and leave `rhs` unused. So this is a total of 13 bytes per AST node.
And again, we don't have to pay for the padding to round up to 16 because
we store in struct-of-arrays format.
I made a further observation: the only kind of data AST nodes need to
store other than the main_token is indexes to sub-expressions. That's it.
The only purpose of an AST is to bring a tree structure to a list of tokens.
This observation means all the data that nodes store are only sets of u32
indexes to other nodes. The other tokens can be found later by the compiler,
by poking around in the tokens array, which again is super fast because it
is struct-of-arrays, so you often only need to look at the token tags array,
which is an array of bytes, very cache friendly.
So for nearly every kind of AST node, you can store it in 13 bytes. For the
rarer AST nodes that have 3 or more indexes to other nodes to store, either
the lhs or the rhs will be repurposed to be an index into an extra_data array
which contains the extra AST node indexes. In other words, no hash table needed,
it's just 1 big ArrayList with the extra data for AST Nodes.
Final observation, no need to have a canonical tag for a given AST. For example:
The expression `foo(bar)` is a function call. Function calls can have any
number of parameters. However in this example, we can encode the function
call into the AST with a tag called `FunctionCallOnlyOneParam`, and use lhs
for the function expr and rhs for the only parameter expr. Meanwhile if the
code was `foo(bar, baz)` then the AST node would have to be `FunctionCall`
with lhs still being the function expr, but rhs being the index into
`extra_data`. Then because the tag is `FunctionCall` it means
`extra_data[rhs]` is the "start" and `extra_data[rhs+1]` is the "end".
Now the range `extra_data[start..end]` describes the list of parameters
to the function.
Point being, you only have to pay for the extra bytes if the AST actually
requires it. There's no limit to the number of different AST tag encodings.
Preliminary results:
* 15% improvement on cache-misses
* 28% improvement on total instructions executed
* 26% improvement on total CPU cycles
* 22% improvement on wall clock time
This is 1/4 items on the checklist before this can actually be merged:
* [x] parser
* [ ] render (zig fmt)
* [ ] astgen
* [ ] translate-c
It now uses the log scope "gpa" instead of "std".
Additionally, there is a new config option `verbose_log` which enables
info log messages for every allocation. Can be useful when debugging.
This option is off by default.
Also known as "Struct-Of-Arrays" or "SOA". The purpose of this data
structure is to provide a similar API to ArrayList but instead of
the element type being a struct, the fields of the struct are in N
different arrays, all with the same length and capacity.
Having this abstraction means we can put them in the same allocation,
avoiding overhead with the allocator. It also saves a tiny bit of
overhead from the redundant capacity and length fields, since each
struct element shares the same value.
This is an alternate implementation to #7854.
comptime direct slice.len increment dodges bounds checking but
we can emit an error for it, at least in the simple case.
- promote original assert to compile-error
- add test case
closes#7810
Adds support for wide, UTF-16, and UTF-32 string literals. If used to initialize
an incomplete array, the same logic as narrow strings is used. Otherwise they
are translated as global "anonymous" arrays of the relevant underlying char type.
A dot is used in the name to ensure the generated names do not conflict with any
other names in the translated program.
For example:
```c
void my_fn() {
const uint32_t *foo = U"foo";
}
```
becomes:
```zig
const @"zig.UTF32_string_2" = [4]c_uint{
'\u{66}',
'\u{6f}',
'\u{6f}',
0,
};
pub export fn my_fn() void {
var foo: [*c]const u32 = &@"zig.UTF32_string_2";
}
```
1. For incomplete arrays with initializer list (`int x[] = {1};`) use the
initializer size as the array size.
2. For arrays initialized with a string literal translate it as an array
of character literals instead of `[*c]const u8`
3. Don't crash if an empty initializer is used for an incomplete array.
4. Add a test for multi-character character constants
Additionally lay some groundwork for supporting wide string literals.
fixes#4831#7832#7842
The CLI gains -flto and -fno-lto options to override the default.
However, the cool thing about this is that the defaults are great! In
general when you use build-exe in release mode, Zig will enable LTO if
it would work and it would help.
zig cc supports detecting and honoring the -flto and -fno-lto flags as
well. The linkWithLld functions are improved to all be the same with
regards to copying the artifact instead of trying to pass single objects
through LLD with -r. There is possibly a future improvement here as
well; see the respective TODOs.
stage1 is updated to support outputting LLVM bitcode instead of machine
code when lto is enabled. This allows LLVM to optimize across the Zig and
C/C++ code boundary.
closes#2845
This temporary patch fixes a segfault caused by miscompilation
by the LLD when generating stubs for initialization of thread local
storage. We effectively bypass TLS in the default panic handler
so that no segfault is generated and the stack trace is correctly
reported back to the user.
Note that, this is linked directly to a bigger issue with LLD
ziglang/zig#7527 and when resolved, we only need to remove the
`comptime` code path introduced with this patch to use the default
panic handler that relies on TLS.
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
string literals and error set types were allocating the ty/val fields of
the anonymous Decl into the owner Decl's arena, rather than the
new anonymous Decl's arena as intended. This caused use of undefined
value later on in the pipeline.
Previously you had to recompile if you wanted to change the log scopes
that get printed. Now, log scopes can be set at runtime, and -Dlog
controls whether all logging is available at runtime.
Purpose here is a nicer development experience. Most likely stage2
developers will always want -Dlog enabled and then pass --debug-log
scopes when debugging particular issues.
ideal capacity is now determined by e.g.
x += x / f
rather than
x = x * b / a
This turns a multiplication into an addition, making it less likely to
overflow the integer. This commit also introduces padToIdeal() which
does saturating arithmetic so that no overflow is possible when
calculating ideal capacity.
closes#7830
The following AST avoids unnecessary derefs now:
* error set decl
* field access
* array access
* for loops: replace ensure_indexable and deref on the len_ptr with a
special purpose ZIR instruction called indexable_ptr_len.
Added an error note when for loop operand is the wrong type.
I also accidentally implemented `@field`.
* fix wrong pointer const-ness when unwrapping optionals
* allow grouped expressions and orelse as lvalues
* ZIR for unwrapping optionals: no redundant deref
- add notes to please don't use rlWrapPtr, this function should be
deleted
* catch and orelse: better ZIR for non-lvalue: no redundant deref;
operate entirely on values. lvalue case still works properly.
- properly propagate the result location into the target expression
* Test harness: better output when tests fail due to compile errors.
* TZIR: add instruction variants. These allow fewer TZIR instructions to
be emitted from zir_sema. See the commit diff for per-instruction
documentation.
- is_null
- is_non_null
- is_null_ptr
- is_non_null_ptr
- is_err
- is_err_ptr
- optional_payload
- optional_payload_ptr
* TZIR: removed old naming convention instructions:
- isnonnull
- isnull
- iserr
- unwrap_optional
* ZIR: add instruction variants. These allow fewer ZIR instructions to
be emitted from astgen. See the commit diff for per-instruction
documentation.
- is_non_null
- is_null
- is_non_null_ptr
- is_null_ptr
- is_err
- is_err_ptr
- optional_payload_safe
- optional_payload_unsafe
- optional_payload_safe_ptr
- optional_payload_unsafe_ptr
- err_union_payload_safe
- err_union_payload_unsafe
- err_union_payload_safe_ptr
- err_union_payload_unsafe_ptr
- err_union_code
- err_union_code_ptr
* ZIR: removed old naming convention instructions:
- isnonnull
- isnull
- iserr
- unwrap_optional_safe
- unwrap_optional_unsafe
- unwrap_err_safe
- unwrap_err_unsafe
- unwrap_err_code
Also, force rewriting of code signature padding at every update so
that we take into account possible section relocs and expansion of
the last preceeding section, e.g., the string table.
This commit also tweak the logic responsible for managing debug lines
in `DebugSymbols`. In particular, in case we update the same function,
we'd previously incorrectly create a cycle adding pointer to the same
`SrcFn` to itself.
Add support for L'<wchar_t>', u'<char16_t>', and U'<char32_t>'. Currently
this just translates wide char literals to \u{NNNNNN} escape codes
(e.g. U'💯' -> '\u{1f4af}')
Another approach would be to emit UTF-8 encoded character literals
directly, but in my opinion this approaches Unicode-complete because it
would require knowledge of which Unicode codepoints have graphical
representations for the emitted source to be readable.
We could also just emit integer literals, but the current method makes
it clear that we have translated a wide character literal and not just
an integer constant.
When codegen ends in failure, we need to manually clean up any fixups
that may have been gathered during that `codegen.generateSymbol` call.
Otherwise, we will end trapping.
Closes#7555
There was still some extra work @Vexu did in that PR having to do with
adding more compile errors and notes for switch expressions, but that
can be added in a follow-up commit.
* Implement error notes
- note: other symbol exported here
- note: previous else prong is here
- note: previous '_' prong is here
* Add Compilation.CObject.ErrorMsg. This object properly converts to
AllErrors.Message when the time comes.
* Add Compilation.CObject.failure_retryable. Properly handles
out-of-memory and other transient failures.
* Introduce Module.SrcLoc which has not only a byte offset but also
references the file which the byte offset applies to.
* Scope.Block now contains both a pointer to the "owner" Decl and the
"source" Decl. As an example, during inline function call, the
"owner" will be the Decl of the caller and the "source" will be the
Decl of the callee.
* Module.ErrorMsg now sports a `file_scope` field so that notes can
refer to source locations in a file other than the parent error
message.
* Some instances where a `*Scope` was stored, now store a
`*Scope.Container`.
* Some methods in the `Scope` namespace were moved to the more specific
type, since there was only an implementation for one particular tag.
- `removeDecl` moved to `Scope.Container`
- `destroy` moved to `Scope.File`
* Two kinds of Scope deleted:
- zir_module
- decl
* astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly
changed to be a reference; this commit fixes it. Fewer ZIR
instructions processed as a result.
- declval_in_module is renamed to declval
- previous declval ZIR instruction is deleted; it was only for .zir
files.
* Test harness: friendlier diagnostics when an unexpected set of errors
is encountered.
* zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst
on the last zir instruction in the block.
Compile log implementation:
* Write to a buffer rather than directly to stderr.
* Only keep track of 1 callsite per Decl.
* No longer mutate the ZIR Inst struct data.
* "Compile log statement found" errors are only emitted when there are
no other compile errors.
-femit-zir and support for .zir source files is regressed. If we wanted
to support this again, outputting .zir would need to be done as yet
another backend rather than in the haphazard way it was previously
implemented.
For parsing .zir, it was implemented previously in a way that was not
helpful for debugging. We need tighter integration with the test harness
for it to be useful; so clearly a rewrite is needed. Given that a
rewrite is needed, and it was getting in the way of progress and
organization of the rest of stage2, I regressed the feature.
In rare occassions, it may happen that string table is allocated free
space preceeding symbol table. This is an error in the eyes of the `dyld`
dynamic loader and thus has to forbidden by the linker.
Fixes two scenarios where @boolToInt() calls were missing:
1. Boolean expression cast to different-size int (char, long, etc)
2. Boolean expression used as parameter for function with int argument
It turns out I was wrong and we can set the page size to the actual
page size used by the target architecture when dividing the binary
into chunks and calculating a hash of each chunk for embedding within
the adhoc code signature. This shaves of a considerable amount of bytes
since we divide the code signature section by at least 2x.
I've also unified the `write` interface of `CodeSignature` struct to
follow that used in every other bit of `MachO`; namely, the functions
now accept a writer instead of a buffer, therefore, there is no need
to manually track where to write each struct field anymore.
* move concurrency primitives that always operate on kernel threads to
the std.Thread namespace
* remove std.SpinLock. Nobody should use this in a non-freestanding
environment; the other primitives are always preferable. In
freestanding, it will be necessary to put custom spin logic in there,
so there are no use cases for a std lib version.
* move some std lib files to the top level fields convention
* add std.Thread.spinLoopHint
* add std.Thread.Condition
* add std.Thread.Semaphore
* new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
* add std.Thread.RwLock
Implementations provided by @kprotty
* CLI: change to -mred-zone and -mno-red-zone to match gcc/clang.
* build.zig: remove the double negative and make it an optional bool.
This follows precedent from other flags, allowing the compiler CLI to
be the decider of what is default instead of duplicating the default
value into the build system code.
* Compilation: make it an optional `want_red_zone` instead of a
`no_red_zone` bool. The default is decided by a call to
`target_util.hasRedZone`.
* When creating a Clang command line, put -mred-zone on the command
line if we are forcing it to be enabled.
* Update update_clang_options.zig with respect to the recent {s}/{} format changes.
* `zig cc` integration with red zone preference.
The macOS version is now obtained by parsing `SystemVersion.plist`.
Test cases added for plist files that date back to '2005 Panther and up
to the recent '2020 Big Sur 11.1 release of macOS.
Thus we are now able to reliably identify 10.3...11.1 and higher.
- drop use of kern.osproductversion sysctl
- drop use of kern.osversion sysctl (fallback)
- drop kern.osversion tests
- add `lib.std.zig.system.detect()`
- add minimalistic parser for `SystemVersion.plist`
- add test cases for { 10.3, 10.3.9, 10.15.6, 11.0, 11.1 }
closes#7569
... and mem.copy operations. Requires slightly larger input buffers than result length. Add helper functions std.mem.alignInBytes and std.mem.alignInSlice.
The NtQueryInformationFile with .FileNormalizedNameInformation is only available in Windows 10 1803 (rs4) and later, however there is probably still another route we can go via ntdll.
Avoid errors if the socket enters the TIME_WAIT state and we need to
re-execute this test before the OS releases it.
This problem was not really a problem before since the accept()-ed
socket was never closed on the server-side.
The overlap between files and sockets is minimal and lumping them
together means supporting only a small subset of the functionalities
provided by the OS.
Moreover the socket and file handles are not always interchangeable: on
Windows one should use Winsock's close() call rather than the one used
for common files.
This will enable code to perform version checks and make it easier to
support multiple versions of Zig.
Within the SemVer implementation, an intermediate value needed to be
coerced to a slice to workaround a comptime bug.
Closes#6466
Zig's format system is flexible enough to add custom formatters. This PR removes the new z/Z format specifiers that were added for printing Zig identifiers and replaces them with custom formatters.
This PR ensures we use system libc headers and system search paths for
framework headers when linking against frameworks and compiling natively
on macOS.
Because this was not set, the `-dynamic-path` argument was not passed
to LLD when linking an ELF binary. This would still generate a valid glibc
binary, however when executing would result in a segfault.
I got this error when re-running the cmake line setting
-DZIG_EXECUTABLE:
```
ninja: warning: multiple rules generate zig1.o. builds involving
this target will not be correct; continuing anyway
[-w dupbuild=warn]
```
But it works fine with make.
* no longer depend on apt.llvm.org, instead we rely on a zig-bootstrap
tarball with pre-built Zig, LLVM, LLD, and Clang. Similar to the
Windows Dev Kit but for Linux. This also makes the script no longer
depend on Docker, libxml2, or GCC.
* remove bash retry logic; it was only needed for flaky apt.llvm.org
and isn't needed for the other resources we download.
* build and upload the experimental std lib automatically generated
docs. langref.html is moved to docs/langref.html because the std lib
docs are in docs/std/.
* the superfluous "zig" directory in $prefix/lib/zig/std/std.zig is
removed from the tarball.
* update_download_page makes a commit in the www.ziglang.org repository
updating data/releases.json so that repository can manage deploys.
* std.ArrayList gains `moveToUnmanaged` and dead code
`ArrayListUnmanaged.appendWrite` is deleted.
* emit_h state is attached to Module rather than Compilation.
* remove the implementation of emit-h because it did not properly
integrate with incremental compilation. I will re-implement it
in a follow-up commit.
* Compilation: use the .codegen_failure tag rather than
.dependency_failure tag for when `bin_file.updateDecl` fails.
C backend:
* Use a CValue tagged union instead of strings for C values.
* Cleanly separate state into Object and DeclGen:
- Object is present only when generating a .c file
- DeclGen is present for both generating a .c and .h
* Move some functions into their respective Object/DeclGen namespace.
* Forward decls are managed by the incremental compilation frontend; C
backend no longer renders function signatures based on callsites.
For simplicity, all functions always get forward decls.
* Constants are managed by the incremental compilation frontend. C
backend no longer has a "constants" section.
* Participate in incremental compilation. Each Decl gets an ArrayList
for its generated C code and it is updated when the Decl is updated.
During flush(), all these are joined together in the output file.
* The new CValue tagged union is used to clean up using of assigning to
locals without an additional pointer local.
* Fix bug with bitcast of non-pointers making the memcpy destination
immutable.
* CBE buffers are only valid during a flush()
* the file is reopened and truncated during each flush()
* CBE now explicitly ignores updateDecl and deleteDecl
* CBE updateDecl is gone
* test case is enabled
In strictly conforming C, identifiers cannot container dollar signs.
However GCC and Clang allow them by default, so translate-c should
handle them. See http://gcc.gnu.org/onlinedocs/cpp/Tokenization.html
I encountered this in the wild in windows.h
Fixes#7585
This broke build scripts that wanted to refer to `exe_dir` or
`install_path`.
There has also been some pushback and discussion on this breaking
change. I think it should be re-evaluated.
This reverts commit a1a1929cf4.
`@setEvalBranchQuota` can be called before the comptime/inline call
stack is created.
For example:
```zig
@setEvalBranchQuota(100);
comptime {
while (true) {}
}
```
Here we need to set the branch_quota before the comptime block creates a
scope for the branch_count.
Previously, the x86_64 backend would remove code for exitlude relocs
if the jump amount were 0. This causes issues as earlier jumps rely on
the jump being present at the same address.
If a static function is defined with no argument list and no prototype
is given, it should be treated as a function that takes no arguments
rather than as a variadic function.
Fixes#7594
To make sure that we don't have to rebuild libc for every case, we now
have a seperate cache directory for the global cache, which remains
the same between test runs.
Also make sure to destory the Compilation before executing a child process,
otherwise the compiler deadlocks. (#7596)
Also adds support for simple operators, like add and subtract.
The intcast and bitcast instruction also have been implemented.
Linking with libc also works, so we can now generate working executables!
`zig build-exe example.zig -fLLVM -lc`:
```
fn add(a: i32, b: i32) i32 {
return a + b;
}
export fn main() c_int {
var a: i32 = -5;
const x = add(a, 7);
var y = add(2, 0);
y -= x;
return y;
}
```
A HashMap has been added which store the LLVM values used in a function.
Together with the alloc and store instructions the following now works:
```
export fn _start() noreturn {
var x: bool = true;
exit();
}
fn exit() noreturn {
unreachable;
}
```
zir.Inst no longer has an `analyzed_inst` field. This is previously how
we mapped ZIR to their TZIR counterparts, however with the way inline
and comptime function calls work, we can potentially have the same ZIR
structure being analyzed by multiple different analyses, such as during
a recursive inline function call. This would cause the `analyzed_inst`
field to become clobbered. So instead, we use a table to map the
instructions to their semantically analyzed counterparts. This will help
with multi-threaded compilation as well.
Scope.Block.Inlining is split into 2 different layers of "sharedness".
The first layer is shared by the whole inline/comptime function call
stack. It contains the callsite where something is being inlined and the
branch count/quota. The second layer is different per function call but
shared by all the blocks within the function being inlined.
Add support for debug dumping br and brvoid TZIR instructions.
Remove the "unreachable code" error. It was happening even for this case:
```zig
if (comptime_condition) return;
bar(); // error: unreachable code
```
We will need smarter logic for when it is legal to emit this compile
error.
Remove the ZIR test cases. These are redundant with other higher level
Zig source tests we have, and maintaining support for ZIRModule as a
first-class top level abstraction is getting in the way of clean
compiler design for the main use case. We will have ZIR/TZIR based test
cases someday to help with testing optimization passes and ZIR to TZIR
analysis, but as is, these test cases are not accomplishing that, and
they are getting in the way.
* scopes properly inherit inlining information
* compile errors of inline function calls are properly attached to the
caller rather than the callee.
- added a test case for this
* --watch still opens a repl if compile errors happen.
Instead of freeing ZIR after semantic analysis, we keep it around so
that it can be used for comptime calls, inline calls, and generic
function calls. ZIR memory is now managed by the Decl arena.
Debug dump() functions are conditionally compiled; only available in
Debug builds of the compiler.
Add a test for an inline function call.
* remove the -Ddump-zir thing. that's handled through --verbose-ir
* rework Fn to have an is_inline flag without requiring any more memory
on the heap per function.
* implement a rough first version of dumping typed zir (tzir) which is
a lot more helpful for debugging than what we had before. We don't
have a way to parse it though.
* keep track of whether the inline-ness of a function changes because
if it does we have to go update callsites.
* add compile error for inline and export used together.
inline function calls and comptime function calls are implemented the
same way. A block instruction is set up to capture the result, and then
a scope is set up that has a flag for is_comptime and some state if the
scope is being inlined.
when analyzing `ret` instructions, zig looks for inlining state in the
scope, and if found, treats `ret` as a `break` instruction instead, with
the target block being the one set up at the inline callsite.
Follow-up items:
* Complete out the debug TZIR dumping code.
* Don't redundantly generate ZIR for each inline/comptime function
call. Instead we should add a new state enum tag to Fn.
* comptime and inlining branch quotas.
* Add more test cases.
* Function calls that happen in a comptime scope get called at
compile-time. We do this by putting the parameters in place as
constant values and then running regular function analysis on the
body.
* Added `Scope.Block.dump()` for debugging purposes.
* Fixed some code to call `identifierTokenString` rather than
`tokenSlice`, making it work for `@""` syntax.
* Implemented `Value.copy` for big integers.
Follow-up issues to tackle:
* Adding compile errors to the callsite instead of the callee Decl.
* Proper error notes for "called from here".
- Related: #7555
* Branch quotas.
* ZIR support?
This code is adapted from pixelherodev paste from IRC
I have added a new fmt option to handle printing slice values ``{v}`` or ``{V}``
While i think it can be made the default, i want your opinion about it
```zig
var slicea = [0]u32{};
var sliceb = [3]u32{ 1, 2, 3 };
std.log.info("Content: {v}", .{slicea});
std.log.info("Content: {v}", .{sliceb});
```
will print:
```
info: Content: []
info: Content: [1, 2, 3]
```
Question:
Should we drop ``{v}`` and make it the default behavior?
* Improve ArrayList & co documentation
- Added doc comments about the validity of references to elements in
an ArrayList and how they may become invalid after resizing operations.
- This should help users avoid footguns in future.
* Improve ArrayListUnmanaged & co's documentation
- Port improved documentation from ArrayList and ArrayList aligned to
their unmanaged counterparts.
- Made documentation for ArrayListUnmanaged & co more inclusive and
up-to-date.
- Made documentation more consistent with `ArrayList`.
* Corrections on ArrayList documentation.
- Remove incorrect/unpreferred wording on ArrayList vs
ArrayListUnmanaged.
- Fix notes about the alignment of ArrayListAligned
- Be more verbose with warnings on when pointers are invalidated.
- Copy+paste a few warnings
* add warning to replaceRange
* revert changes to append documentation
The reason this is required is for two reasons: 1) the libc
targeting `aarch64` macOS is slightly newer than that targeting
`x86_64`, and 2) `OSAtomic.h` uses relative imports rather than
system-wide imports for accompanying headers which clearly is an
oversight on Apple's part. Until such time when `libkern` headers
between `x86_64` and `aarch64` are identical, this will require a
manual intervention to duplicate the relevant headers between the
respective architectures.
This commit adds default search paths for system frameworks
on macOS while also adding `-isysroot` for OS versions at least BigSur.
Since BigSur (11.0.1), neither headers nor libs exist in standard
root locations (`/usr/include`, `/System/Library/Frameworks`). Instead, they
are now exclusively part of the installed developer toolchain (either
via XCode.app or CLT), and specifying `-isysroot` allows us to keep
using universal search paths such as `/System/Library/Frameworks` while
only changing the include flag from `-iframework` to
`-iframeworkwithsysroot`.
restore "Comply with semantic versioning pre-release format"
restore "stage2: SemVer compliance for development builds"
restore "Remove 'g' prefix from commit hash in Zig semver"
This reverts commit d96d8639e5.
This reverts commit e8810f5794.
This reverts commit 9afe5859a3.
git describe is used for version string creation, but it had to be
reverted in commit 69da6ba because it was broken in CI builds.
Azure Pipelines and Drone perform shallow clones by default.
This change reconfigures them to fetch history and tags. It adds tens of
seconds, which is negligible compared to overall build and test time.
Related: #6466, #6509, #7601
macOS requires the debug symbols to either be part of the intermediate
object file `whatever.o` or a companion `whatever.dSym` bundle. The
former case seems ill-suited for our needs since it subscribes to
the old-fashioned compilation strategy using intermediate compilation
units; the latter is what we need however on macOS the debug symbols
unlike in Elf are not part of the final artefact; rather they sit
next to it in its own Mach-O file.
One exception will be treatment of the __LINKEDIT segment which
will be handled separately inside MachO directly since it doesn't
include any sections.
This patch introduces the following new things:
Types:
- inferred_alloc
- This is a special value that tracks a set of types that have been stored
to an inferred allocation. It does not support most of the normal type queries.
However it does respond to `isConstPtr`, `ptrSize`, `zigTypeTag`, etc.
- The payload for this type simply points to the corresponding Value
payload.
Values:
- inferred_alloc
- This is a special value that tracks a set of types that have been stored
to an inferred allocation. It does not support any of the normal value queries.
ZIR instructions:
- store_to_inferred_ptr,
- Same as `store` but the type of the value being stored will be used to infer
the pointer type.
- resolve_inferred_alloc
- Each `store_to_inferred_ptr` puts the type of the stored value into a set,
and then `resolve_inferred_alloc` triggers peer type resolution on the set.
The operand is a `alloc_inferred` or `alloc_inferred_mut` instruction, which
is the allocation that needs to have its type inferred.
Changes to the C backend:
* Implements the bitcast instruction. If the source and dest types
are both pointers, uses a cast, otherwise uses memcpy.
* Tests are run with -Wno-declaration-after-statement. Someday we can
conform to this but not today.
In ZIR form it looks like this:
```zir
fn_body main { // unanalyzed
%0 = dbg_stmt()
=>%1 = alloc_inferred()
%2 = declval_in_module(Decl(add))
%3 = deref(%2)
%4 = param_type(%3, 0)
%5 = const(TypedValue{ .ty = comptime_int, .val = 1})
%6 = as(%4, %5)
%7 = param_type(%3, 1)
%8 = const(TypedValue{ .ty = comptime_int, .val = 2})
%9 = as(%7, %8)
%10 = call(%3, [%6, %9], modifier=auto)
=>%11 = store_to_inferred_ptr(%1, %10)
=>%12 = resolve_inferred_alloc(%1)
%13 = dbg_stmt()
%14 = ret_type()
%15 = const(TypedValue{ .ty = comptime_int, .val = 3})
%16 = sub(%10, %15)
%17 = as(%14, %16)
%18 = return(%17)
} // fn_body main
```
I have not played around with very many test cases yet. Some interesting
ones that I want to look at before merging:
```zig
var x = blk: {
var y = foo();
y.a = 1;
break :blk y;
};
```
In the above test case, x and y are supposed to alias.
```zig
var x = if (bar()) blk: {
var y = foo();
y.a = 1;
break :blk y;
} else blk: {
var z = baz();
z.b = 1;
break :blk z;
};
```
In the above test case, x, y, and z are supposed to alias.
I also haven't tested with `var` instead of `const` yet.
This is the same as the previous commit but for Value instead of Type.
Add `Value.castTag` and note that it is preferable to call than
`Value.cast`. This matches other abstractions in the codebase.
Added a convenience function `Value.Tag.create` which really cleans up
the callsites of creating `Value` objects.
`Value` tags can now share payload types. This is in preparation for
another improvement that I want to do.
Add `Type.castTag` and note that it is preferable to call than
`Type.cast`. This matches other abstractions in the codebase.
Added a convenience function `Type.Tag.create` which really cleans up
the callsites of creating `Type` objects.
`Type` payloads can now share types. This is in preparation for another
improvement that I want to do.
The current API does not allow the user to distinguish between EOF and
an empty line. Reader.readUntilDelimiterOrEof() gets this API right so
update readUntilDelimiterOrEofAlloc() to match it. Returning an optional
here additionally makes calling this in a loop much cleaner.
Remove readUntilDelimiterOrEofArrayList() as it no longer needed to
implement readUntilDelimiterOrEof() and has the same API issues
described without a clear way to fix them.
Add -Werror -pedantic -std=c89 to make sure our outputted C code is
squeaky clean.
Allow garbage to be printed to stderr for Execution tests because
wasmtime is printing a bogus warning, causing the CI to fail.
* Module: improve doc comments
* C backend: improve const-correctness
* C backend: introduce renderTypeAndName
* C backend: put `static` on functions when appropriate
* C backend: fix not handling errors in genBinOp
* C backend: handle more IR instructions
- alloc, store, boolean comparisons, ret_ptr
* C backend: call instruction properly stores its result
* test harness: ensure execution tests have empty stderr
Example version: 0.8.0-dev.460+g81b343a16
From a semantic versioning perspective, development builds will now be
considered distinct from a tagged release. The number of commits added
since the last tag is included in the pre-release component.
Updates #6466
before executing child process. This fixes a deadlock when the test
wanted to obtain the same lock on compiler_rt.o that was held by the
process building the test binary itself.
This is a partial merge of #6750. I took the Posix code paths and
dropped the Windows code paths, and then did the improvements noted in
the comments.
The Windows implementation is still TODO.
* read directly into the ArrayList buffers.
* respect max_output_bytes
* std.ArrayList:
- make `allocatedSlice` public.
- add `unusedCapacitySlice`.
I removed the Windows implementation of this stuff; I am doing a partial
merge of LemonBoy's patch with the understanding that a later patch can
add the Windows implementation after it is vetted.
Keep polling until there are enough open handles, if the child process
terminates closing the handles or explicitly closes them we just quit
polling and wait for the process handle to signal the termination
condition.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
Previous commit broke the tests for non-Windows because we were
intending to change the cwd when running the child process. However, for
Windows we don't support passing a directory handle for cwd when
spawning child processes yet. However on Linux we do. This commit
reverts the previous one but then fixes things for all systems by
passing both cwd_dir and cwd to the child process.
This broke compiling with -Dstage1 on Windows, because it added the
cpp_source_files to the compilation multiple times.
This reverts commit c81ae52ee0.
These tests asserted there were no args passed to the test binary, but
now there is an arg intentionally passed to the test binary, so the test
case needed to be updated.
This branch introduces a new kind of test into the stage2 test harness:
Zig code that compiles into C code with the C backend, and then the
resulting C code gets run and output compared against the expected
result.
This branch also implements extern functions in the frontend so that we
can have a "hello world" C backend test that passes.
The addition of `addDeclErr` introduced a memory leak at every call
site, and I also would like to push back on having more than 1
compilation error per `Decl`.
This reverts commit 1634d45f1d.
This is not in fact safe to use with GeneralPurposeAllocator as GPA
requires align(page_size) but raw_c_allocator provides only
@alignOf(std.c.max_align_t).
We were violating the POSIX standard which resulted in a deadlock on
musl v1.1.24 on aarch64 alpine linux, uncovered with the new ThreadPool
usage in the stage2 compiler.
std.os execv functions that accept an Allocator parameter are removed
because they are footguns. The POSIX standard does not allow calls to
malloc() between fork() and execv() and since it is common to both
(1) call execv() after fork() and (2) use std.heap.c_allocator,
Programmers are encouraged to go through the `std.process` API
instead, causing some dissonance when combined with `std.os` APIs.
I also slapped a big warning message on all the relevant doc comments.
Previously casting a bool to an int would result in the following Zig code:
@intCast(c_int, @bitCast(i1, @intCast(u1, @boolToInt(b))));
This is incorrect if `b` is true, since bitcasting a `u1` with the value 1
to an `i1` will result in the value -1. Instead, generate the following code:
@as(c_int, @boolToInt(b));
Since @boolToInt returns a `u1`, this is only disallowed if the destination
type is one-bit and signed, which can only happen if it's a bitfield
(currently not supported by translate-c)
Previously Zig would need to recompile runtime libs if you changed the
values of --strip or -O. Now, unless the `debug_compiler_runtime_libs`
flag is set (which is currently not exposed to the CLI), Zig will always
choose ReleaseFast or ReleaseSmall for compiler runtime libraries.
When the main application chooses ReleaseFast or ReleaseSmall, that
value is propagated to compiler runtime libraries. Otherwise a decision
is made based on the target, which is currently ReleaseSmall for
freestanding WebAssembly and ReleaseFast for everything else.
Ultimately the purpose of this commit is to have Debug and ReleaseSafe
builds of applications still get optimized builds of, e.g. libcxx and
libunwind, as well as to spend less time unnecessarily rebuilding compiler
runtime libraries.
Making the enum type share the scope with the parent union means every
declaration "bleeds" into the enum scope.
Let's mint a fresh empty scope for the enum type.
Thanks to @Vexu for the test case.
Closes#7532
* rename is_compiler_rt_or_libc to skip_linker_dependencies
and set it to `true` for all sub-Compilations. I believe
this resolves the deadlock we were experiencing on Drone
CI and on some users' computers. I will remove the CI workaround in
a follow-up commit.
* enabling TSAN automatically causes the Compilation to link against
libc++ even if not requested, because TSAN depends on libc++.
* add -fno-rtti flags where appropriate when building TSAN objects.
Thanks Firefox317 for pointing this out.
* TSAN support: resolve all the undefined symbols. We are still seeing
a dependency on __gcc_personality_v0 but will resolve this one in a
follow-up commit.
* static libs do not try to build libc++ or libc++abi.
* split std.ResetEvent into:
- ResetEvent - requires init() at runtime and it can fail. Also
requires deinit().
- StaticResetEvent - can be statically initialized and requires no
deinitialization. Initialization cannot fail.
* the POSIX sem_t implementation can in fact fail on initialization
because it is allowed to be implemented as a file descriptor.
* Completely define, clarify, and explain in detail the semantics of
these APIs. Remove the `isSet` function.
* `ResetEvent.timedWait` returns an enum instead of a possible error.
* `ResetEvent.init` takes a pointer to the ResetEvent instead of
returning a copy.
* On Darwin, `ResetEvent` is implemented using Grand Central Dispatch,
which is exposed by libSystem.
stage2 changes:
* ThreadPool: use a single, pre-initialized `ResetEvent` per worker.
* WaitGroup: now requires init() and deinit() and init() can fail.
- Add a `reset` function.
- Compilation initializes one for the work queue in creation and
re-uses it for every update.
- Rename `stop` to `finish`.
- Simplify the implementation based on the usage pattern.
Previous wording made it seem like any signed or floating-point value would be allowed at comptime, whereas negative values do not work with `%`, and negative integers do not work with `/`.
* Truncate user and group ids
Calls to `getuid`, `getgid` and their `eid` variants fail to compile on
64bit Linux systems because the return value of the syscall is of
`usize` and needs to be truncated to fit the size of `uid_t` that is 32
bit.
Thanks to @FireFox317 for figuring this out in Zig's Discord channel!
* Add a regression test for user and group ids
* Replace @truncate with @intCast
This should be safe because `uid_t` will be 32-bit.
* Add missing import for getauxval
* Add missing package names
* Revert "Add missing import for getauxval"
This reverts commit 38f93dc89effdf657f2b81a56b96527ce4083f52.
* Skip user and group test if builtin.link_libc
* stage2: add @TypeOf
* stage2: discriminate on what type of @builtinCall in nodeMayNeedMemoryLocation
* merge upstream into my stash
* add type equality to make easier to test and defer free the types
* remove addDeclErr, I dont know why I added it, its from a different branch that im working on
* add tests
* update error message to match stage1
* use ComptimeStringMap and update which nodes don't need memory from vexu's suggestions
* fix typo
Co-authored-by: Veikka Tuominen <git@vexu.eu>
* make @TypeOf(single_arg) go to .typeof zir inst and add test for that
* unioninit, as, reduce change mayneedmemorylocation
Co-authored-by: Veikka Tuominen <git@vexu.eu>
C compiler intrinsics can only appear as part of a function call. When called
they are implicitly cast to a function pointer; treat this as a non-null
pointer so that it emits as a regular Zig function call.
Put `pub usingnamespace @import("std").c.builtins;` at the top of translated
C files so that they will have access to builtin functions defined there.
Fixes#6707
We generally get away with atomic primitives, however a lock is required
around the refresh function since it traverses the Node graph, and we
need to be sure no references to Nodes remain after end() is called.
However, adding a twist where `findFreeSpace` accepts a `SegmentCommand`
as argument meaning we want to look for free space specifically within
that segment and nowhere else.
Firstly, we preallocate segments offset and sizes, and then
when adding sections, we find the free space within each segment.
Currently, this applies to any segment that is not __LINKEDIT
segment since this requires special treatment.
Everybody gets what they want!
* AT_RANDOM is completely ignored.
* On Linux, MADV_WIPEONFORK is used to provide fork safety.
* On pthread systems, `pthread_atfork` is used to provide fork safety.
* For systems that do not have the capability to provide fork safety,
the implementation falls back to calling getrandom() every time.
* If madvise is unavailable or returns an error, or pthread_atfork
fails for whatever reason, it falls back to calling getrandom() every
time.
* Applications may choose to opt-out of fork safety.
* Applications may choose to opt-in to unconditionally calling
getrandom() for every call to std.crypto.random.fillFn.
* Added `std.meta.globalOption`.
* Added `std.os.madvise` and related bits.
* Bumped up the size of the main thread TLS buffer. See the comment
there for justification.
* Simpler hot path in TLS initialization.
* get rid of the pointless fences
* make seed_len 16 instead of 32, which is accurate since it was
already padding the rest anyway; now we do 1 pad instead of 2.
* secureZero to clear the AT_RANDOM auxval
* add a flag root source files can use to disable the start code. This
is in case people want to opt out of the initialization when they
don't depend on it.
std.crypto.random
* cross platform, even freestanding
* can't fail. on initialization for some systems requires calling
os.getrandom(), in which case there are rare but theoretically
possible errors. The code panics in these cases, however the
application may choose to override the default seed function and then
handle the failure another way.
* thread-safe
* supports the full Random interface
* cryptographically secure
* no syscall required to initialize on Linux (AT_RANDOM)
* calls arc4random on systems that support it
`std.crypto.randomBytes` is removed in favor of `std.crypto.random.bytes`.
I moved some of the Random implementations into their own files in the
interest of organization.
stage2 no longer requires passing a RNG; instead it uses this API.
Closes#6704
ad05509 introduced a fix for the wrong problem, the logic to align the
start of main_thread_tls_buffer was already there but was flawed.
Fix it for good and avoid wasting too many bytes for alignment purposes.
auto_hash must be extra careful when hashing integers whose bit size is
not a multiple of 8 as, when reinterpreted with mem.asBytes, may contain
undefined non-zero bits too.
Rather than directly pasting the output from the
target-details-generator tool, we have to look at the diff line by line
and decide how to incorporate the changes.
The baseline abstraction is something that Zig provides.
The changes to x86 appear to be incorrect. After LLVM commit
3ad09fd03c51823aeb0bcbd7898aada33e9228d6 the CPU features are stored in
a different data layout in their tables, and so we need to update the
target details extraction tool to match.
- addrinfo: addr and canonname are switched (wrong layout)
- addrinfo, Flock, msghdr struct: use proper c_xxx type instead of fixed size. it should help using struct on all architectures supported by openbsd
Don't cut any corner and properly run the type trough every single step
even though it has no fields (or, better, the sum of the size of all its
fields is zero).
Fix the logic to consider an explicit non-zero-sized tag enough to treat
the type as sized.
Closes#7451
Xnu's sigaction() only supports fetching a limited set of sa_flags, test
SA_SIGINFO instead of SA_RESETHAND as that's supported everywhere.
Add another check to make sure SA_RESETHAND works.
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
extension("a.") now returns "." instead of "".
This matches both Python and Node.js standard library behavior as well
as my personal opinion on how this function should be defined.
Apologies for missing this in the code review.
The steps to repro this issue are:
zig build-obj hello.zig -target x86_64-windows-msvc
zig build-exe hello.obj -target x86_64-windows-msvc --subsystem console
-lkernel32 -lntdll
What was happening is that the main Compilation added a work item to
produce kernel32.lib. Then it added a sub-Compilation to build zig's
libc, which ended up calling a function with extern "kernel32", which
caused the sub-Compilation to also try to produce kernel32.lib. The main
Compilation and sub-Compilation do not coordinate about the set of
import libraries that they will be trying to build, so this caused a
deadlock.
This commit solves the problem by disabling the extern "foo" feature
from working when building compiler_rt or libc. Zig's linker code is now
responsible for putting the appropriate import libs on the linker line,
if any for compiler_rt and libc.
Related: #5825
Cache exposes BinDigest.
Compilation gains a set of a BinDigest for every C/C++ source file. We
detect when the same source/flags have already been added and emit a
compile error. This prevents a deadlock in the caching system.
Closes#7308
Previously, when choosing the local cache directory, if there was no
root source file, an explicitly chosen path, or other clues, zig would
choose cwd + zig-cache/ as the local cache directory.
This can be problematic if Zig is invoked with the CWD set to a
read-only directory, or a directory unrelated to the actual source files
being compiled. In the real world, we see this when using `zig cc` with
CGo, which for some reason changes the current working directory to the
read-only go standard library path before running the C compiler.
This commit conservatively chooses to use the global cache directory
as the local cache directory when there is no other reasonable choice,
and no longer will rely on the cwd path to choose a local cache directory.
As a reminder, the --cache-dir CLI flag and ZIG_LOCAL_CACHE_DIR
environment variable are available for overriding the decision. For the
zig build system, it will always choose the directory that build.zig is
+ zig-cache/.
Closes#7342
* Field global_cache_root was added to Builder struct along with
mandatory argument for build_runner.zig. Logic for using the custom
global cache was also added.
* The arguments --cache-dir and --global-cache-dir are no longer passed
directly through to build_runner.zig and are instead only passed through the
mandatory cache_root and global_cache_root arguments.
In C, enums are represented as signed integers, so casting from an enum to an integer
should use the "cast integer to integer" translation code path. Previously it used the
"cast enum to generic non-enum" code path, because enums were not being treated as integers.
Ultimately this can produce zig code that fails to compile if the destination type does not
support the full range of enum values (e.g. translated C code that casts an enum value to an
unsigned integer would fail to compile since enums are signed integers, and unsigned integers
cannot represent the full range of values that signed ones can).
One interesting thing that came up during testing is that the implicit enum-to-int cast that
occurs when an enum is used in a boolean expression was parsed as an (int) by some versions of
the zig compiler, and an (unsigned int) cast by others. Specifically, the following code:
```c
enum Foo {Bar, Baz};
// ...
enum Foo foo = Bar;
if (0 || foo) {
// do something
}
```
When tested on MacOS, Linux, and Windows using a compiler built from the Windows Zig Compiler
Dev Kit, the above code would emit a cast to c_uint:
`if (false or (@bitCast(c_uint, @enumToInt(foo)) != 0)) {}`
However when tested on Windows with a Zig compiler built using MSVC, it produces:
`if (false or (@bitCast(c_int, @enumToInt(foo)) != 0)) {}`
In this particular case I don't think it matters, since a c_int and c_uint will have the same
representation for zero, but I'm not sure if this is ultimately the result of
implementation-defined behavior or something else.
Because of this, I added explicit casts in the `translate_c.zig` tests, to ensure that the
emitted zig source exactly matches across platforms. I also added a behavior test in
`run_translated_c.zig` that uses the old implicit casts from `translate_c.zig` to ensure
that the emitted Zig code behaves the same as the C code regardless of what cast is used.
Previously, --name would only be inferred if there was exactly 1 C
source file or exactly 1 object. Now it will be inferred if there is at
least one of either.
This allows to have multiple instances of `zig build` at the same
time. For example when you have a long running `zig build run` and
then want to run `zig build somethingelse`.
Closes#7356
I did this as a patch to the source rather than passing flags so that
it would intentionally be reverted when we update to the next release of
mingw-w64. At this time if any warnings are still emitted we should find
out why and make sure upstream is aware of the problem.
ring.timeout() to queue a IORING_OP_TIMEOUT operation
ring.timeout_remove() to queue a IORING_OP_TIMEOUT_REMOVE operation
io_uring_prep_timeout() to prep a IORING_OP_TIMEOUT sqe
io_uring_prep_timeout_remove() to prep a IORING_OP_TIMEOUT_REMOVE sqe
Let's a void any kind of compilation/LLVM errors for niche targets such
as AVR/MSP430 or ARM v6m. By not exporting any atomic builtin anymore
the user is free to provide their own implementation (that disable the
IRQs) or to provide the --single-threaded switch and forget about this.
passthrough mode does not mean always exit - it just means to pass
through stdio and exit if the child process exits, without doing any
special error reporting.
This commit adds:
ZIG_LOCAL_CACHE_DIR corresponding to --cache-dir
ZIG_GLOBAL_CACHE_DIR corresponding to --global-cache-dir
ZIG_LIB_DIR corresponding to --override-lib-dir
The main use case is for `zig cc` where we are bound by clang's CLI
options and need alternate channels to pass these configuration options.
Previously, when mixing Zig and C/C++ code for windows-gnu targets, zig
would get codeview format but the C/C++ code would not get any debug
info. Now, C/C++ code properly emits debug info in codeview format and
everything just works.
The code is a bit fragile so it was causing CI failures on x86_64-macos.
Also the patch up code is only needed for aarch64-macos, so we were
doing unnecessary work.
The readme now links to the wiki for instructions on building from
source. I plan to make many updates to the wiki soon as I start to
provide tarballs for working on the compiler.
restore cmake to be capable of figuring out the zig version
restore config.h and config.zig. config.h is used to detect whether we
should propagate cmake configuration information to build.zig; however
it can be overridden with -Dstatic-llvm.
fix not passing -DZIG_LINK_MODE with zig build.
when using the cmake build path, build.zig no longer tries to call
llvm-config. Instead it relies 100% on the LLVM_LIBRARIES cmake variable.
build.zig logic reworked and simplified.
The main idea here is that there are now 2 ways to get a stage1 zig
binary:
* The cmake path. Requirements: cmake, system C++ compiler, system
LLVM, LLD, Clang libraries, compiled by the system C++ compiler.
* The zig path. Requirements: a zig installation, system LLVM, LLD,
Clang libraries, compiled by the zig installation.
Note that the former can be used to now take the latter path.
Removed config.h.in and config.zig.in. The build.zig script no longer is
coupled to the cmake script.
cmake no longer tries to determine the zig version. A build with cmake
will yield a stage1 zig binary that reports 0.0.0+zig0. This is going to
get reverted.
`zig build` now accepts `-Dstage1` which will build the stage1 compiler,
and put the stage2 backend behind a feature flag.
build.zig is simplified to only support the use case of enabling LLVM
support when the LLVM, LLD, and Clang libraries were built by zig. This
part is probably sadly going to have to get reverted to make package
maintainers happy.
Zig build system addBuildOption supports a couple new types.
The biggest reason to make this change is that the zig path is an
attractive option for doing compiler development work on Windows. It
allows people to work on the compiler without having MSVC installed,
using only a .zip file that contains Zig + LLVM/LLD/Clang libraries.
* Remove function parseAnyBaseInt.
* Replace calls to parseAnyBaseInt to calls to std.fmt.parseUnsigned
with radix 0.
* Replace calls to parseInt where the type is unsigned with calls to
parseUnsigned. Note that the functionality of these arguments haven't
changed, they still retain the original radix specified.
When the tag type is not a zero-sized type (eg. `enum(i32)`) we
absolutely need to avoid constant-folding this values. Doing so masked
any invalid input and, since the folding was not even applied
consistently, introduced some hard to catch errors.
Fill in the correct value instead of leaving everything uninitialized.
This problem can be noticed in behavior/union.zig but it's masked by
some other "optimization" kicking in at the wrong time, the following
commits will address that.
Use case:
zig build-exe non_existent_file.zig
Previous behavior:
error.FileNotFound, followed by an error return trace
Behavior after this commit:
error: unable to read non_existent_file.zig: FileNotFound
(end of stderr, exit code 1)
This turns AllErrors.Message into a tagged union which now has the
capability to represent both "plain" errors as well as source-based
errors (with file, line, column, byte offset). The "no entry point found"
error has moved to be a plain error message.
This then allows for proper resolution of names via runpath search
path list, i.e., `-rpath @loader_path` will correctly resolve
to `@rpath/libxxx.dylib (...)` in the linked binary.
This commit version-gates appending `-syslibroot` flag to lld. This
is predicated upon the fact that for versions of macOS lower than
11, lld would fail to find and link against frameworks with this
flag specified.
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
This way, in the very situation where a function has a return type
an error union such as `anyerror!void` but doesn't have any erroneous
paths, calling `@errorName` on the unpacked error (which will never
be triggered) will not trip up the static analyzer.
* Fix floating point parsing on BE systems
* Load the appropriate endian.h files for macOS and BSD
* Add endian definition for Windows and extra check for ldshape selection
* Fix endian macro definition for macOS
Apparently their macros are defined without a leading __.
* Define new macro for endian checking purposes
This is gross and I really do not like the lack of standardization
around this part, but what can I do?
This commit addresses comments suggesting a cleaner approach
at converting an `extern` struct to its byte representation using
`mem.asBytes`, and to use `meta.eql` in place of more fragile
`mem.eql(u8, ...)` for comparison of two `extern` structs.
Thanks LemonBoy!
Applications supplying their own custom stack to pthread_create are not
allowed to free the allocated memory after pthread_join returns as,
according to the specification, the thread is not guaranteed to be dead
after the join call returns.
Avoid this class of problems by avoiding the use of a custom stack
altogether, let pthread handle its own resources.
Allocations made on the child stack are now done on the C heap.
Thanks @semarie for noticing the problem on OpenBSD and suggesting a
fix.
Closes#7275
If we enter the `if` because `comp.disable_c_depfile` is false the `man`
object has no manifest and calling .final on it will trip an assertion.
Closes#7096
commit 9d1816111d used the "output path"
as the path for passing shared library artifact paths to the Zig CLI.
For Windows, this was incorrect because it would pass the .dll instead
of the .lib file. This commit passes the "output lib path" instead,
which makes it pass the .lib path in case of a .dll on Windows.
This way the linker does not complain and say, "bad file type. Did you
specify a DLL instead of an import library?"
See #5854
Some tiny tweaks too:
* Use `wasm-freestanding-musl` instead of `wasm32-freestanding-musl`,
making it pointer-size-agnostic.
* Fix trying to build non-existent wasm musl start files.
Positional shared library arguments were not being detected as causing
dynamic linking, resulting in invalid linker lines. LLD did not have an
error message for this when targeting x86_64-linux but it did emit an
error message when targeting aarch64-linux, which is how I noticed the
problem.
This surfaced an error having to do with fifo.pipe() in the cat example
which I did not diagnose but solved the issue by doing the revamp that
was already overdue for that example.
It appears that the zig-window project was exploiting the previous
behavior for it to function properly, so this prompts the question, is
there some kind of static/dynamic executable hybrid that the compiler
should recognize? Unclear - but we can discuss that in #7240.
* it is now -fcompiler-rt and -fno-compiler-rt to override the (quite
reasonable) default of bundling compiler-rt only for executables and
dynamic libraries.
- the build.zig API is still called bundle_compiler_rt however it is
now an optional bool instead of a bool. leaving it as `null` means
to use the compiler default.
* renamed some internal identifiers to make the source more readable
* additionally support -fcompiler-rt when doing build-obj for ELF files
since that target already supports linking multiple objects into one.
- includes an error message when attempting this for non-ELF. in the
future this could additionally be supported with a more advanced
implementation that does not rely on the linker.
* properly populate the linker cache hash
I spent a long time working on this data structure, and I still think
it's a neat idea, but it has no business being in the std lib.
I'm aware of the few remaining references to SegmentedList that exist in
the std lib, but they are dead code, and so I'm leaving the dead
references as a clue that the code is dead. Cleaning up dead code will
be a separate effort that involves code coverage tools to make sure we
find it all.
std-lib-orphanage commit: 2c36a7894c689ecbaf63d5f489bb0c68773410c4
closes#7190
* Switch to using unicode when parsing the command line on windows
* Apply changes by LemonBoy and *hopefully* fix tests on MIPs
Co-authored-by: LemonBoy <LemonBoy@users.noreply.github.com>
* Fix up next and skip
* Move comment to more relevant place
Co-authored-by: LemonBoy <LemonBoy@users.noreply.github.com>
ZigTypeIdOptional types have a different way of specifying their payload
value depending on whether the child type is a pointer or not (plus some
other special cases).
Fixes#7251
stderr_mutex may still be held when we reach the signal handler, grab
our own stderr handle to print the error messages and avoid deadlocking.
Closes#7247
Before this commit, the branch regressed this case:
zig build-lib foo.zig -dynamic
readelf -d libfoo.so
x000000000000000e (SONAME) Library soname: [libfoo.so]
Now it works again, but it preserves the property that using `zig cc`
has the SONAME off by default to match C compilers.
As discussed with LemonBoy, it makes more sense to have the `-soname`
an opt-in option rather than opt-out, which is especially true if
we are to align `zig cc` with `cc` to a greater degree.
fork() on Linux/sparc64 seems to return its result in two registers,
with %o0 always holding the current process' PID, and the parent/child
status returned in %o1. Add some glue code to convert those into
the libc-style return value.
Make gen_const_ptr_struct_recursive aware of the possible presence of
some trailing padding by always bitcasting the pointer to its expected
type.
Not an elegant solution but makes LLVM happy and is consistent with how
the other callsites are handling this case.
Fixes#5398
* always align the stack to 16. I saw an instance on x86_64 linux where
it was needed.
* detect at runtime if being interpreted by a dynamic loader and if so
avoid clobbering the fs register.
Recent versions of cmake complain about it:
<<
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
>>
We don't require anything from version 2.8.5, and version 2.8.12 was
released in 2011, so it is very unlikely that anyone still uses 2.8.5.
Find the effective ELF load address in dl_iterate_phdr by computing the
difference between the in-memory phdr and its p_vaddr specified in the
ELF file.
This makes the dl_iterate_phdr test pass and restores the stack traces.
Conflicts:
lib/std/dynamic_library.zig (fixed in this commit)
src/all_types.hpp
src/codegen.cpp
src/link.cpp
src/main.cpp
Will manually apply the diffs to these deleted files to the new zig
code in a followup commit.
The previous clone() definition was written for 32bit PPC targets,
trying to execute it on a PPC64 machine lead to countless crashes.
Use the clone() definition kindly lifted from Musl libc.
Turn the FSM parser into a linear one so that's easier to implement new
features and/or more error checking without adding more and more states.
Functionally-speaking the two parsers are at feature parity.
* stage2: Make zig cc more verbose
Make `zig cc` print more info from Clang itself and from our own linker
invocation, this is needed for CMake to properly discover all the
include directories and library search paths.
Closes#7110
* Update `update_clang_options`
* Typo fixes
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
* add more abosolutes
* added wrong files
* adding 2 tests and changing the function signatures because of lazy analysis not checking them
* fix a bug that got uncovered by lazy eval
* Add compile error when using WASI with openDirAbsolute and accessAbsolute
* typo
Transforming scalars to non-adjacent form shrinks the number of
precomputations down to 8, while still processing 4 bits at a time.
However, real-world benchmarks show that the transform is only
really useful with large precomputation tables and for batch
signature verification. So, do it for batch verification only.
https://github.com/cfrg/draft-irtf-cfrg-hash-to-curve
This is quite an important feature to have since many other standards
being worked on depend on this operation.
Brings a couple useful arithmetic operations on field elements by the way.
This PR also adds comments to the functions we expose in 25519/field
so that they can appear in the generated documentation.
Make the code used to calculate the variable slot index into the frame
match what's done during the structure layout calculation.
Prevents a few nasty LLVM errors when such types are passed around.
See ziglang/fetch-them-macos-headers#2 for more details. The path
forward looks like one of the following:
* Ony provide headers for the oldest supported macOS (currently 10.13
but soon to be bumped to 10.14).
* Provide headers for multiple versions, and select based on the Zig
target OS version range minimum.
* Don't try to provide macOS headers.
If we don't tackle the version problem, we would have to re-introduce
the ability to detect and depend on native system headers if we wanted
to support C/C++ code that used newer OS definitions.
This patch also adds support for `#include <mach/mach_time.h>`.
Also related: #5236
This is andrewrk's patch from ziglang#6724 (rebased)
CMake: Fix dependency problem
I don't know whether the error was expected cmake behavior or a bug.
This change seems to fix the issue. See ziglang#6724 for details.
We currently have ciphers optimized for performance, for
compatibility, for size and for specific CPUs.
However we lack a class of ciphers that is becoming increasingly
important, as Zig is being used for embedded systems, but also as
hardware-level side channels keep being found on (Intel) CPUs.
Here is ISAPv2, a construction specifically designed for resilience
against leakage and fault attacks.
ISAPv2 is obviously not optimized for performance, but can be an
option for highly sensitive data, when the runtime environment cannot
be trusted.
I have an alternative set of windows bindings I'm working on: https://github.com/marler8997/zig-os-windows. So I'm declaring my wWinMain function with my own HINSTANCE type rather than the one from std.os.windows. This change allows start to call wWinMain using any pointer type.
* Move leb128 out of debug and remove trivial *mem functions as discussed in #5588
* Turns out one of the *Mem functions was used by MachO. Replaced with trivial use of FixedBufferStream.
Make it behave like the read() wrapper for unix systems.
Reading the whole buffer breaks some use-cases like buffered readers
over sockets.
Closes#7121
The mismatch between the argument slot type in the frame structure and
the one used in the store operation made the generated code write
garbage over the nearby fields.
Fixes#7104
This intentionally diverges from the unix dirname command, as well as
Python and Node.js standard libraries, which all have this edge case
return the input path, unmodified. This is a footgun, and nobody should
have ever done it this way.
Even the man page contradicts the behavior. It says:
"strip last component from file name". Now consider, if you
remove the last item from an array of length 1, then you
have now an array of length 0. After you strip the last component, there
should be no components remaining. Clearly, returning the input parameter
unmodified in this case does not match the documented behavior. This is
my justification for taking a stand on this API design.
closes#6746closes#6727closes#6584closes#6592closes#6602
The code tried to be too smart and skipped the equality (returning true)
if the payload type was zero-sized.
This optimization is completely wrong when the union payload is a
metatype!
Fixes#7047
raise the timeout to 100ms to be sure that if it fails (timeout is returned) it is due to a real problem.
the test shouldn't be longer: it will wait more time only on failure.
these new files are generated by recursively including:
sysexits.h
mach/mach.h
sys/attr.h
sys/mount.h
crt_externs.h
execinfo.h
all of which are depended on by LLVM libraries.
This is a trivial implementation that just does a or[xor] loop.
However, this pattern is used by virtually all crypto libraries and
in practice, even without assembly barriers, LLVM never turns it into
code with conditional jumps, even if one of the parameters is constant.
This has been verified to still be the case with LLVM 11.0.0.
This reverts commit 663f0b399c.
The behavior appears to be inconsistent between running locally and on
the CI.
I suspect it could be based on what vector CPU features are available.
As documented in the comment right above the finalization function,
Gimli can be used as a XOF, i.e. the output doesn't have a fixed
length.
So, allow it to be used that way, just like BLAKE3.
With the simple rule that whenever we have or will have 2 similar
functions, they should be in their own namespace.
Some of these new namespaces currently contain a single function.
This is to prepare for reduced-round versions that are likely to
be added later.
It is now possible to force linking with system linker `ld` instead
of the LLVM `lld` linker when building natively on the target. This
can be done at each stage by specifying `--system-linker-hack` flag,
and can be useful on platforms where `lld` fails to operate properly
such as macOS 11 Big Sur on ARM64 where every binary/dylib is expected
to be codesigned.
Some example invocations for each stage of compilation of Zig
toolchain:
```
cmake .. -DCMAKE_PREFIX_PATH=/path/to/llvm -DSYSTEM_LINKER_HACK=1
```
```
build/zig build test --system-linker-hack
```
```
build/zig build --prefix $(pwd)/stage2 -Denable-llvm
--system-linker-hack
```
```
build/zig build-exe hello.zig --system-linker-hack
```
Make the code easier for the optimizer to work with and introduce a fast
path for ASCII sequences.
Introduce a benchmark harness to start tracking the performance of ops
on utf8.
Do the alignment dance by ourselves whenever posix_memalign is not
available.
Don't try to use malloc as it has too many edge cases, figuring out
whether a block of memory is manually aligned by the mechanism above or
is directly coming from malloc becomes too hard to be valuable.
Comparisons with absolute epsilons are usually useful when comparing
numbers to zero, for non-zero numbers it's advised to switch to relative
epsilons instead to obtain meaningful results (check [1] for more
details).
The new API introduces approxEqAbs and approxEqRel, where the former
aliases and deprecated the old `approxEq`, allowing the user to pick the
right tool for the job.
The documentation is meant to guide the user in the choice of the
correct alternative.
[1] https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/
There's no guarantee for the kernel definition to be ABI compatible with
the libc one (and vice versa).
There's also no guarantee of ABI compatibility between musl/glibc.
Fun, isn't it?
The version we were using in CI is now getting quite old, and we
want to make sure that the code we produce is compatible with
current versions of Wasmtime.
With this commit, the function tries to use more efficient syscalls, and
then falls back to non-positional reads.
The motivating use case for this change is to support something like the
following:
try io.getStdOut().writeFileAll(dest_file, .{});
We are checking that two identical, constant values, are stored at
different addresses.
But sharing a unique location doesn't look like something the compiler
wouldn't do.
It may make more sense to check that a const variable and a mutable
variable set to the same value have different addresses.
We read and write bytes directly from the state, but in the init
function, we potentially endian-swap them.
Initialize bytes in native format since we will be reading them
in native format as well later.
Also use the public interface in the "permute" test rather than an
internal interface. The state itself is not meant to be accessed directly,
even in tests.
On Darwin, according to the man pages for setrlimit(), when adjusting
max number of open fds, the reported hard max by getrlimit() is only
theoretical, while the actual maximum, set in the kernel, is hardcoded
in the header file. Therefore, the reported max has to be adjusted
as `min(OPEN_MAX, lim.max)`.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
This commit fixes linking issue on macOS 11 BigSur by appending
a prefix path to all lib and framework search paths known as
`-syslibroot`.
The reason this is needed is that in macOS 11, the system libraries
and frameworks are no longer readily available in the filesystem.
Instead, the new macOS ships with a built-in dynamic linker cache
of all system-provided libraries, and hence, when linking with either
`lld.ld64` or `ld64`, it is required to pass in `-syslibroot [dir]`.
The latter can usually be obtained by invoking `xcrun --show-sdk-path`.
With this commit, Zig will do this automatically when compiling natively
on macOS. However, it also provides a flag `-syslibroot` which can be
used to overwrite the automtically populated value.
To summarise, with this change, the user of Zig is not required to
generate and append their own syslibroot path. Standard invocations
such as `zig build-exe hello.zig` or `zig build` for projects will
work out of the box. The only missing bit is `zig cc` and `zig c++`
since the addition of the `-syslibroot` option would be a mismatch
between the values provided by `clang` itself and Zig's wrapper.
Both `offset` and `len` are `off_t`.
Like the rest of the std lib we assume that `_FILE_OFFSET_BITS == 64`
is always true, so that `off_t` is a `u64`.
When passing to 32-bit kernels, we split these into two `u32`
parameters.
BLAKE2 includes the expected output length in the initial state.
This length is actually distinct from the actual output length
used at finalization.
BLAKE2b-256/128 is thus not the same as BLAKE2b-128.
This behavior can be a little bit surprising, and has been "fixed"
in BLAKE3.
In order to support this, we may want to provide an option to set the
length used for domain separation.
In Zig, there is another reason to allow this: we assume that the
output length is defined at comptime.
But BLAKE2 doesn't have a fixed output length. For an output length that
is not known at comptime, we can't take the full block size and
truncate it due to the reason above.
What we can do now is set that length as an option to get the correct
initial state, and truncate the output if necessary.
Leverage result location semantics for X25519 like we do everywhere
else in 25519/*
Also add the edwards25519->curve25519 map by the way since many
applications seem to use this to share the same key pair for encryption
and signature.
The code was using the length of the local symbols, which also
includes the null symbol. Fix this by using the offset table instead,
which only keeps track of the symbols that end up in the got.
A flag meant to catch recursively-defined types was never reset, leading
the compiler to generate wrong answers when asked for its
type/alignment.
Closes#6850
Intel keeps changing the latency & throughput of the aes* and clmul
instructions every time they release a new model.
Adjust `optimal_parallel_blocks` accordingly, keeping 8 as a safe
default for unknown data.
It seems that Apple has finally got rid of the 32bit versions of
`fstat` and `fstatat`, and instead, only 64bit versions are available
on BigSur and Apple Silicon.
The tweak in this commit is required to make Zig stage1 compile on
BigSur + aarch64.
std.os.accept already wants to allow null, which matches `man 3p accept`:
> address Either a null pointer, or a pointer to a sockaddr structure
> where the address of the connecting socket shall be re‐
> turned.
>
> address_len Either a null pointer, if address is a null pointer, or a
> pointer to a socklen_t object which on input specifies the
> length of the supplied sockaddr structure, and on output
> specifies the length of the stored address.
Fixes ziglang#6832.
Gives a ~40% speedup on x86_64.
However, the generic code remains faster on aarch64.
This is still processing only one block at a time for now.
I'm pretty confident that processing more blocks per round
will eventually give a substantial performance improvement on
all platforms with vector units.
The bcrypt function intentionally requires quite a lot of CPU cycles
to complete.
In addition to that, not having its full state constantly in the
CPU L1 cache causes a massive performance drop.
These properties slow down brute-force attacks against low-entropy
inputs (typically passwords), and GPU-based attacks get little
to no advantages over CPUs.
The NaCl constructions are available in pretty much all programming
languages, making them a solid choice for applications that require
interoperability.
Go includes them in the standard library, JavaScript has the popular
tweetnacl.js module, and reimplementations and ports of TweetNaCl
have been made everywhere.
Zig has almost everything that NaCl has at this point, the main
missing component being the Salsa20 cipher, on top on which NaCl's
secretboxes, boxes, and sealedboxes can be implemented.
So, here they are!
And clean the X25519 API up a little bit by the way.
of https://github.com/BarabasGitHub/zig into
BarabasGitHub-improve-windows-networking
Conflicts:
lib/std/os.zig
This commit resolves conflicts with the changes to std.os which removed
the EAGAIN error handling interactions with the event loop. The solution
to the conflict was to apply EAGAIN => return error.WouldBlock into the
improved windows networking branch.
The fix for #6715 introduced a new compiler error when attempting to use
wWinMain as the application entry-point.
The Windows API often relies on implicit casts between signed and
unsigned variables. In this case, wWinMain returns an INT despite the
fact this value is intended to feed into ExitProcess, which expects a
UINT, so I've restored the bitcast from #5613.
In this commit, the code samples in the language reference have been changed to
use `std.testing.expect` rather than `std.debug.assert` when they are
written in `test` code. This will teach Zig learners best practices when
they write their own test code.
Not all uses of `std.debug.assert` have been replaced. There are examples where
using `assert` fits the context of the sample.
Using `std.debug.assert` in test code can lead to errors if running tests in
ReleaseFast mode. In ReleaseFast mode, the `unreachable` in `assert` is
undefined behavior. It is possible that `assert` always causes `zig test` to
pass thus possibly leading to incorrect test code outcomes. The goal is to
prevent incorrect code from passing test cases.
Closes#5836
Calling @panic made the executable ~30x times bigger, use a simple
`abort()` and let the user figure out what went wrong.
Supporting ARMv6 (and earlier?) platforms is not a priority.
Closes#6676
Explicit and implicit integer casts on vector types are now supported
and follow the same rules as their scalar counterparts.
Implicit float casts are accidentally supported, `@floatCast` is still
not vector-aware.
* Smaller startup sequence for ppc64
* Terminate the frame-pointer chain when executing _start
* Make the stack traces work on ppc64
* Make the stack traces coloured on ppc64, some ioctls numbers are
different and the whole set of constants should be audited.
mimics the duplication of strings in `Builder` for `Pkg`. This
ensures the lifetime of the memory backing strings in a `Pkg`
struct and the `Pkg.dependencies` slice is not shorter than the
`Builder` that the data is associated with.
The new defaults that came in with
644400054c are nice, however, it is still
possible that someone knows their inputs are always small and wants to
use the simpler implementations. We keep the default to make the choice
at runtime, but expose the linear functions in the public interface of
std.mem.
Also improved the doc comments.
- use `PascalCase` for all types. So, AES256GCM is now Aes256Gcm.
- consistently use `_length` instead of mixing `_size` and `_length` for the
constants we expose
- Use `minimum_key_length` when it represents an actual minimum length.
Otherwise, use `key_length`.
- Require output buffers (for ciphertexts, macs, hashes) to be of the right
size, not at least of that size in some functions, and the exact size elsewhere.
- Use a `_bits` suffix instead of `_length` when a size is represented as a
number of bits to avoid confusion.
- Functions returning a constant-sized slice are now defined as a slice instead
of a pointer + a runtime assertion. This is the case for most hash functions.
- Use `camelCase` for all functions instead of `snake_case`.
No functional changes, but these are breaking API changes.
* move the opaque section to after struct, enum, union, and add
hyperlinks
* improve the introduction of the zig build system. don't link to the
wiki.
* update to the latest zig init-exe example code
* rename headers to avoid redundant words such as "zig"
* simplify example code
Unlike glibc and musl, MinGW provides no libssp symbols leading to
countless compile errors if FORTIFY_SOURCE is defined.
Add a (incomplete) implementation of libssp written in Zig so that
linking succeeds.
Closes#6492
Pay close attention to the RHS type alignment when rendering an
assignment op as it may differ from the LHS pointer one.
This problem was noticed when debugging a CI failure in #6648: due to
sheer luck the misalignment caused a segfault on macos that was also
reproduced locally.
I tried to write a small test case but it turned out to be a daunting
task as I couldn't manage to trigger the problem consistently (and stop
the optimizer from simplifying everything). Patches welcome.
* remove GetModuleHandleA from kernel32.zig. use of A functions
considered harmful.
* make it a compile error to expose WinMain instead of wWinMain. same
thing.
* start code declares wWinMainCRTStartup instead of WinMainCRTStartup
when it has the choice.
`DefaultCsprng` is documented as a cryptographically secure RNG.
While `ISAAC` is a CSPRNG, the variant we have, `ISAAC64` is not.
A 64 bit seed is a bit small to satisfy that claim.
We also saw it being used with the current date as a seed, that
also defeats the point of a CSPRNG.
Set `DefaultCsprng` to `Gimli` instead of `ISAAC64`, rename
the parameter from `init_s` to `secret_seed` + add a comment to
clarify what kind of seed is expected here.
Instead of directly touching the internals of the Gimli implementation
(which can change/be architecture-specific), add an `init()` function
to the state.
Our Gimli-based CSPRNG was also not backtracking resistant. Gimli
is a permutation; it can be reverted. So, if the state was ever leaked,
future secrets, but also all the previously generated ones could be
recovered. Clear the rate after a squeeze in order to prevent this.
Finally, a dumb test was added just to exercise `DefaultCsprng` since
we don't use it anywhere.
HMAC is a generic construction, so we allow it to be instantiated
with any hash function.
In practice, HMAC is almost exclusively used with MD5, SHA1 and SHA2,
so it makes sense to define some shortcuts for them.
However, defining `HmacBlake2s256` is a bit weird (and why
specifically that one, and not other hash functions we also support?).
There would be nothing wrong with that construction, but it's not
used in any standard protocol and would be a curious choice.
BLAKE2 being a keyed hash function, it doesn't need HMAC to be used as
a MAC, so that also doesn't make it a good example of a possible hash
function for HMAC.
This commit doesn't remove the ability to use a Hmac(Blake2s256) type
if, for some reason, applications really need this, but it removes
HmacBlake2s256 as a constant.
* extract logic into a `os_can_execve` and use it in the other place
that we execve
* outdent some code by introducing `run_or_test` variable
* delete unnecessary and wasteful memory management logic
* better error message for when execve fails
* add comment to explain why we do not execve for `zig test`
* std.Target.standardDynamicLinkerPath: macOS has a dynamic linker
* no need to override the default dynamic linker in the macos
CrossTarget initialization in the tests
* in getExternalExecutor, when validating the dynamic linker path, take
into account the standard dynamic linker path.
This commit enables stage2 end-to-end tests to run natively on macOS
(where and when applicable). Since QEMU on macOS doesn't support
the same type of architecture emulation as it does on linux (i.e.,
there is no `qemu-x86_64` for instance), this commit ensures that we
specify a path to dynamic linker on macOS (`/usr/lib/dyld`) which
is then checked for existence in `std.CrossTarget.getExternalExecutor()`
function, and if exists, we can run the test natively.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
This is slightly slower but makes our verification function compatible
with batch signatures. Which, in turn, makes blockchain people happy.
And we want to make our users happy.
Add convenience functions to substract edwards25519 points and to
clear the cofactor.
On macOS, a .tbd ("text-based dylib definition") file is a shared library
stub, allowing symbols to be defined only once for all the architectures
the library was compiled for.
.tbd files can be linked like .dylib files.
Brings a 30% speed boost on x86_64 even though we still process only
one block at a time for now.
Only enabled on x86_64 since the non-vectorized implementation seems
to currently perform better on some architectures (at least on aarch64).
But the non-vectorized implementation still gets a little speed boost
as well (~17%) with these changes.
* Add an optimized squaring routine under the `sqr` name.
Algorithms for squaring bigger numbers efficiently will come in a
PR later.
* Fix a bug where a multiplication was done twice if the threshold for
the use of Karatsuba algorithm was crossed. Add a test to make sure
this won't happen again.
* Streamline `pow` method, take a `Const` parameter.
* Minor tweaks to `pow`, avoid bit-reversing the exponent.
* std.fs.Dir.readFile: add doc comments to explain what it means when
the returned slice has the same length as the supplied buffer.
* introduce readSmallFile / writeSmallFile to abstract over the
decision to use symlink or file contents to store data.
Conflicts:
src/clang.zig
Master branch renamed an enum; this branch gave it an explicit tag type
and explicitly initialized values. This commit combines the changes
together.
Conflicts:
cmake/Findllvm.cmake
The llvm11 branch changed 10's to 11's and master branch added the
"using LLVM_CONFIG_EXE" help message, so the resolution was to merge
these changes together.
I also added a check to make sure LLVM is built with AVR enabled, which
is no longer an experimental target.
* std.fs.File.copyRange and copyRangeAll return u64 instead of usize -
the returned value is how much of the `len` is transferred, so the
types should match. This removes the need for an `@intCast`.
* fix typo that removed a subtraction
* Fix the size of codegen.AnyMCValue which gave me a compile error when
I tried to build self-hosted for i386-linux.
* restore the coercion to u64 of syms_sect.sh_info. We want to make
sure the multiplication happens with 64 bits and not the smaller type
used by the ELF format.
* fix another offset parameter in link/Elf.zig to be u64 instead of usize
* add a nice little TODO note to help out Jakub
* FmtError already has FileTooBig in it; we just need to return it.
zig0 is only used for building objects, thus it has no options like
`build-exe/obj`. But when cross-compiling, we have a working zig
compiler on the host, thus we need to pass `build-obj` to the zig compiler.
On some distros (e.g. Void Linux) the release field of the tsname
struct may contain an underscore followed by a revision number at the
end. (e.g. 5.8.12_2).
* Correctly scan all the exponent bits, this caused the incorrect result
to be computed for exponents being powers of two.
* Allocate enough limbs to make llmulacc stop whining.
This reverts commit 70f3767903.
After discussion, I can see the value provided here, specifically with
avoiding the footgun of defer { suspend { free(@frame()); } }.
However the doc comments are updated to explain the semantics directly,
rather than basing them on the behavior of another programming language.
It was regressed in 2 ways from the merge of #6250:
* it was not being enabled by default when the target OS is native.
* we were testing the libfoo.so file path existence with bogus format
string ('{}' instead of '{s}') and so it ended up being something
like "libstd.HashMap(K,V,...).Entry.so" instead of "libfoo.so". Using
{} rather than {s} is a footgun, be careful!
Previous functionality is now restored.
closes#6523
Zig supports both Linux and Windows natively. Those are the main use
cases we are focusing on. Happy to accept mingw patches if they are
non-invasive, but we will not be hampering progress on the main use
cases with failing mingw CI builds.
Before this it was trying to copy all the files from the zig-cache dir
to the output dir, but in the current compiler architecture the cache
dir is also used for internal compiler files.
This commit refactors free space analysis (and VM analysis) somewhat.
This is still far from perfect, but at least it's not using any
hardcoded values.
This commit also reorganizes different flushing bits here and there,
so that bit common to Exe and Obj are put in the common path, while
Exe-specific are behind an appropriate switch/if statement.
* Convert draft to generate all relevant segments and sections in right places
* Do not prealloc space in text blocks until we can NOP
* Write out LC_LOAD_DYLINKER command
* Add LC_LOAD_DYLIB command in order to specify to load libSystem
* Redo update decl exports (similar to Elf globals, globals need to be contiguous in
memory)
Decouples SQE queueing and SQE prepping methods to allow for non-sequential
SQE allocation schemes as suggested by @daurnimator.
Adds essential SQE prepping methods from liburing to reduce boilerplate.
Removes non-essential .link_with_next_sqe() and .use_registered_fd().
Implemented following Knuth's "Evaluation of Powers" chapter in TAOCP,
some extra complexity is needed to make sure there's no aliasing and
avoid allocating too many limbs.
A brief example to illustrate why the last point is important:
consider 10^123, since 10 is well within the limits of a single limb we
can safely say that the result will surely fit in:
⌈log2(10)⌉ bit * 123 = 492 bits = 7 limbs
A naive calculation using only the number of limbs yields:
1 limb * 123 = 123 limbs
The space savings are noticeable.
This reverts commit c8b4cc2ff9.
This includes many C++ standard library headers:
#include <algorithm>
#include <chrono>
#include <ctime>
#include <iomanip>
#include <ios>
#include <iostream>
#include <memory>
#include <sstream>
#include <string>
#include <tuple>
#include <type_traits>
#include <vector>
which adds more than a second of compile time for each file that
includes the header:
ir.cpp before: 8.041s
ir.cpp after: 6.847s
Ensures that the wakeup flag is read after the tail pointer has been
written. It's important to use memory load acquire semantics for the
flags read, otherwise the application and the kernel might not agree on
the consistency of the wakeup flag, leading to I/O starvation.
Refs: 6768ddcc56
Refs: https://github.com/axboe/liburing/issues/219
Non-leaf functions now work. Combined with simple parameters and return
types, this allows more complicated programs than Hello World to be
correctly compiled.
Performance increases from ~400 MiB/s to 450 MiB/s at the expense of
extra code. Thus, aggregation is disabled on ReleaseSmall.
Since the multiplication cost is significant compared to the reduction,
aggregating more than 2 blocks is probably not worth it.
with respect to std.builtin.link_libc.
The commit 27e008eb29 did not solve the
problem because although it got std.builtin.link_libc to be true for
compiler_rt.zig and c.zig, it had other unintentional side effects which
broke the build for -lc -target foo-linux-musl.
This commit introduces a new flag to Compilation to allow setting this
comptime flag to true without introducing other side effects to
compilation and linking.
This was causing the Dir.readLink test to fail for me locally with error.Unexpected NTSTATUS=0xc0000022. Not sure if PRIVILEGE_NOT_HELD is actually possible or not.
I want to take the design of this in a different direction. I think this
abstraction is too high level. I want to start bottom-up.
std-lib-orphanage commit 179ae67d61455758d71037434704fd4a17a635a9
comment reproduced here:
This is so that compiler_rt and libc.zig libraries know whether they
will eventually be linked with libc. They make different decisions
about what to export depending on whether another libc will be linked
in. For example, compiler_rt will not export the __chkstk symbol if it
knows libc will provide it, and likewise c.zig will not export memcpy.
This is convenient for debugging purposes, as well as simplifying the
caching system since executable basenames will not conflict with their
corresponding object files.
https://github.com/ziglang/std-lib-orphanage/
This code is not used by anything else in the standard library or by the
compiler or any of its tools and therefore it's a great candidate
to be maintained by a third party.
* change some {} to be {s} to gain type safety
* fix libraries being libfoo.lib instead of foo.lib for COFF
* when linking mingw-w64, add the "always link" libs so that we
generate DLL import .lib files for them as the linker code relies on.
* COFF LLD linker does not support -r so we do a file copy as an
alternative to the -r thing that ELF linking does.
I will file an issue for the corresponding TODO upon merging this
branch, to look into an optimization that possibly elides this copy
when the source and destination are both cache directories.
* add a CLI error message when trying to link multiple objects into one
and using COFF object format.
Showcase that Zig can be a great option for high performance cryptography.
The AEGIS family of authenticated encryption algorithms was selected for
high-performance applications in the final portfolio of the CAESAR
competition.
They reuse the AES core function, but are substantially faster than the
CCM, GCM and OCB modes while offering a high level of security.
AEGIS algorithms are especially fast on CPUs with built-in AES support, and
the 128L variant fully takes advantage of the pipeline in modern Intel CPUs.
Performance of the Zig implementation is on par with libsodium.
Comment reproduced here:
If we're linking libc, some naughty applications may have
registered atexit handlers which we really do not want to
run in the fork child. I caught LLVM doing this and it
caused a deadlock instead of doing an exit syscall. In
the words of Avril Lavigne, "Why'd you have to go and
make things so complicated?"
When we get a cache hit for a stage1 compilation, we need to know about
some of the flags such as have_winmain or have_dllmain to know which
subsystem to infer during linking.
To do this, we append a hex-encoded byte into the intentionally-dangling
symlink which contains the cache hash digest rather than a filename. The
hex-encoded byte contains the flags we need to infer the subsystem
during linking.
Before:
gimli-hash: 120 MiB/s
gimli-aead: 130 MiB/s
After:
gimli-hash: 195 MiB/s
gimli-aead: 208 MiB/s
Also fixes in-place decryption by the way.
If the input & output buffers were the same, decryption used to fail.
Return on decryption error in the benchmark to detect similar issues
in future AEADs even in non release-fast mode.
stage1 was unable to parse ranges whose starting point was written in
binary/octal as the first dot in '...' was incorrectly interpreted as
decimal point.
stage2 forgot to reset the literal type to IntegerLiteral when it
discovered the dot was not a decimal point.
I've only stumbled across this bug because zig fmt keeps formatting the
ranges without any space around the ...
Previously if a LinearFifo was empty and discard was called
an unsigned overflow would occur. However it is safe to perform
this overflow as a bitwise & operation with 0xFFFFFFFFFFFFFF is a noop
* std.log: still print error messages in ReleaseSmall builds.
- when start code gets an error code from main, it uses std.log.err
to report the error. this resulted in a test failure because
ReleaseSmall wasn't printing `error: TheErrorCode` when an error
was returned from main. But that seems like it should keep working.
So I changed the std.log defaults. I plan to follow this up with a
proposal to change the names of and reduce the quantity of the
log levels.
* warning emitted when using -femit-h when using stage1 backend; fatal
log message when using -femit-h with self-hosted backend (because the
feature is not yet available)
* fix double `test-cli` build steps in zig's build.zig
* update docgen to use new CLI
* translate-c uses `-x c` and generates a temporary basename with a
`.h` extension. Otherwise clang reports an error.
* --show-builtin implies -fno-emit-bin
* restore the compile error for using an extern "c" function without
putting -lc on the build line. we have to know about the libc
dependency up front.
* Fix ReleaseFast and ReleaseSmall getting swapped when passing the
value to the stage1 backend.
* correct the zig0 CLI usage text.
* update test harness code to the new CLI.
This is helpful on systems with multiple LLVM installations.
For example, OpenBSD ships with LLVM in the base system, but without
support for extra targets.
A full LLVM version can be installed using the ports system, but even
when it is, `cmake` is not going to pick it up unless `/usr/local` is
explicitly added to the `cmake` search paths.
Having the full `llvm-config` path printed on error is helpful to
understand that the detected LLVM version was not the expected one.
These CLI options are now forwarded to the stage1 backend.
We're not going to support the -mllvm CLI option any longer. As a
compromise, we unconditionally tell LLVM to output intel x86 syntax when
using -femit-asm.
Simplify stage1 logic; it no longer has the concept of an output
directory. --output-dir is no longer a valid CLI option. cmake uses
the `-femit-bin=[path]` option.
Note the changes to test/cli.zig. This breaks the CLI API that Godbolt
is using so we're going to want to open a PR to help them upgrade to the
new CLI for the upcoming Zig 0.7.0 release.
This cleans up how the CLI parses and handles -E, -S, and -c.
Compilation explicitly acknowledges when it is being used to do C
preprocessing.
-S is properly translated to -fno-emit-bin -femit-asm but Compilation
does not yet handle -femit-asm.
There is not yet a mechanism for skipping the linking step when there is
only a single object file, and so to make this work we have to do a file
copy in link.flush() to copy the file from zig-cache into the output
directory.
These APIs allow one to write code that is agnostic of whether it is
using an ArrayHashMap or a HashMap, which can be valuable. Specify
intent precisely: if you only need the count of the items, it makes
sense to have a function for that.
Normally when using LLD to link, Zig uses a file named "lld.id" in the
same directory as the output binary which contains the hash of the link
operation, allowing Zig to skip linking when the hash would be unchanged.
In the case that the output binary is being emitted into a directory which
is externally modified - essentially anything other than zig-cache - then
this flag would be set to disable this machinery to avoid false positives.
* Better defaults when using -fno-LLVM
* Fix compiler_rt and libc static libraries were getting a .zig
extension instead of .a extension.
* when using the stage1 backend, put the object file next to the
stage1.id file in the cache directory. this prevents an object file
from polluting the cwd when using zig from the CLI.
* prefix with the message level
* if the scope is not default, also prefix with the scope
This makes the stack trace test pass, with no changes to the
test case, because errors returned from main() now print
`error: Foo` just like they do in master branch.
* Don't try to generate C header files yet since it will only cause a
crash saying the feature is unimplemented.
* Rename the CLI options for release modes to use the `-O` prefix to
match C compiler precedent. Options are now `-ODebug`,
`-OReleaseFast`, `-OReleaseSafe`, `-OReleaseSmall`. The optimization
mode matches the enum tags of std.builtin.Mode. It is planned to, at
some point, rename std.builtin.Mode to std.builtin.OptimizationMode
and modify the tags to be lower case to match the style convention.
- Update build.zig code to support this new CLI.
* update std.zig.binNameAlloc to support an optional Version and update
the implementation to correctly deal with dynamic library version
suffixes.
* Reorganize crypto/aes in order to separate parameters, implementations and
modes.
* Add a zero-cost abstraction over the internal representation of a block,
so that blocks can be kept in vector registers in optimized implementations.
* Add architecture-independent aesenc/aesdec/aesenclast/aesdeclast operations,
so that any AES-based primitive can be implemented, including these that don't
use the original key schedule (AES-PRF, AEGIS, MeowHash...)
* Add support for parallelization/wide blocks to take advantage of hardware
implementations.
* Align T-tables to cache lines in the software implementations to slightly
reduce side channels.
* Add an optimized implementation for modern Intel CPUs with AES-NI.
* Add new tests (AES256 key expansion).
* Reimplement the counter mode to work with any block cipher, any endianness
and to take advantage of wide blocks.
* Add benchmarks for AES.
Password hashing functions are not general-purpose KDFs, and KDFs
don't have to satisfy the same properties as a PHF.
This will allow fast KDFs such as the HKDF construction to be in a
category of their own, while clarifying what functions are suitable
for using passwords as inputs.
A simple empty main with evented-io would not quit, because some
threads were still waiting to be resumed (by the os). The os.write to
the eventfd only wakes up one thread and thus there are multiple writes
needed to wake up all the other threads.
As part of this:
* add std.process.cleanExit. closes#6395
- use it in several places
* adjust the alignment of text in `zig build --help` menu
* Cache: support the concept of "unhit" so that we properly keep track
of the cache when we find out using the secondary hash that the cache
"hit" was actually a miss. Use this to fix false negatives of caching
of stage1 build artifacts.
* fix not deleting the symlink hash for stage1 build artifacts causing
false positives.
* implement support for Package arguments in stage1 build artifacts
* update and add missing usage text
* add --override-lib-dir and --enable-cache CLI options
- `--enable-cache` takes the place of `--cache on`
* CLI supports -femit-bin=foo combined with --enable-cache to do an
"update file" operation. --enable-cache without that argument
will build the output into a cache directory and then print the path
to stdout (matching master branch behavior).
* errors surfacing from main() now print "error: Foo" instead of
"error: error.Foo".
* add CLI support for verbose debugging options
* implement building compiler_rt and libc static archive using stage1
C++ backend
* add function_sections and link_libcpp to root cache hash.
* cleanups to use the new Directory.join method.
* Package supports being initialized directly and cleaned up create()
method.
* fix classifyFileExt incorrectly saying .zir is .zig. Thanks
@Rocknest!
* unify updateSubCompilation implementations
HeapAllocator stores the pointer returned by HeapAlloc right after the
data block and, after the recent allocator refactoring, the space for
this pointer was not taken into account in the calculation of the final
block size.
Fixes#5830
* move stage2.cpp code into zig0.cpp for simplicity
* add -ftime-report and some more CLI options to stage2
* stage2 compites the llvm cpu features string
* classifyFileExt understands more file extensions
* correction to generateBuiltinZigSource using the wrong allocator
(thanks dbandstra!)
* stage2 is now able to build hello.zig into hello.o using stage1 as a
library however it fails linking due to missing compiler-rt
* remove dead code
* simplify zig0 builtin.zig source
* fix not resolving builtin.zig source path causing duplicate imports
* fix stage1.h not being valid C code
* fix stage2.h not being valid C code
This takes the place of `zig builtin`. This is an improvement over the
command because now the generated source will correctly show LinkMode
and OutputMode, whereas before it was always stuck as Static and Obj,
respectively.
Deleted 16,000+ lines of c++ code, including:
* an implementation of blake hashing
* the cache hash system
* compiler.cpp
* all the linking code, and everything having to do with building
glibc, musl, and mingw-w64
* much of the stage1 compiler internals got slimmed down since it
now assumes it is always outputting an object file.
More stuff:
* stage1 is now built with a different strategy: we have a tiny
zig0.cpp which is a slimmed down version of what stage1 main.cpp used
to be. Its only purpose is to build stage2 zig code into an object
file, which is then linked by the host build system (cmake) into
stage1. zig0.cpp uses the same C API that stage2 now has access to,
so that stage2 zig code can call into stage1 c++ code.
- stage1.h is
- stage2.h is
- stage1.zig is the main entry point for the Zig/C++
hybrid compiler. It has the functions exported from Zig, called
in C++, and bindings for the functions exported from C++, called
from Zig.
* removed the memory profiling instrumentation from stage1.
Abandon ship!
* Re-added the sections to the README about how to build stage2 and
stage3.
* stage2 now knows as a comptime boolean whether it is being compiled
as part of stage1 or as stage2.
- TODO use this flag to call into stage1 for compiling zig code.
* introduce -fdll-export-fns and -fno-dll-export-fns and clarify
its relationship to link_mode (static/dynamic)
* implement depending on LLVM to detect native target cpu features when
LLVM extensions are enabled and zig lacks CPU feature detection for
that target architecture.
* C importing is broken, will need some stage2 support to function
again.
- Moves fs.rename functions to fs.renameAbsolute to match other functions outside of fs.Dir
- Adds fs.Dir.rename that takes two paths relative to the given Dir
- Adds fs.rename that takes two separate Dir's that the given paths are relative to (for renaming across directories without having to make the second path relative to a single directory)
- Fixes FileNotFound error return in std.os.windows.MoveFileExW
- Returns error.RenameAcrossMountPoints from renameatW
+ Matches the RenameAcrossMountPoints error return in renameatWasi/renameatZ
Here's the doc comment from the commit:
For one example of why this is handy, consider the case of building musl libc.
We keep a lock open for each of the object files in the form of a file descriptor
until they are finally put into an archive file. This is to allow a zig-cache
garbage collector to run concurrently to zig processes, and to allow multiple
zig processes to run concurrently with each other, without clobbering each other.
This code is disabled until #6361 is implemented (getrlimit/setrlimit
are not yet added to the standard library).
This implementation tries to do the right thing (TM) by treating the
sign as part of the number itself, therefore the alignment parameter
applies to both the sign and the digits.
In other words the format string `{:>4}` with -1 as input will not
output `- 1` but ` -1`.
And let's default to right alignment for everything as that's what users
want, especially when printing numbers. Many implementations use
different defaults for numeric vs non-numeric types, let's strive for a
consistent behaviour here.
* caching system: use 16 bytes siphash final(), there was a bug in the
std lib that wasn't catching undefined values for 18 bytes. fixed in
master branch.
* fix caching system unit test logic to not cause error.TextBusy on windows
* port the logic from stage1 for building glibc shared objects
* add is_native_os to the base cache hash
* fix incorrectly freeing crt_files key (which is always a reference to
global static constant data)
* fix 2 use-after-free in loading glibc metadata
* fix memory leak in buildCRTFile (errdefer instead of defer on arena)
This reverts commit 40cb712d13.
Thanks to Ava & Luna of Lavatech, we don't need to resort to this, they
have graciously given zig a SourceHut instance to use that gives us 8GB
RAM.
Before merging, do this for every item in the file:
* solve the issue, or
* convert the task to a github issue and update the comment
to link to the issue (and remove "TODO" text from the comment).
Then delete the file.
Related: #363
Drew won't give us enough RAM for stage1 to build stage2. We'll still
have freebsd builds available on releases but we're going to lose
freebsd CI testing for master branch builds until we fully switch over
to stage2 (and have lower memory usage).
Let me know if anyone wants to run a SourceHut instance and give zig
access to run on slightly more powerful machines. We need about 8 GiB
RAM to run the CI test suite for now.
After we're fully self hosted I expect to re-enable this.
The API is pretty specific to the implementationt details of the
self-hosted compiler. I don't want to have to independently support
and maintain this as part of the standard library, and be obligated
to not make breaking changes to it with changes to the implementation of
stage2.
This is not strictly necessary but it increases the likelihood of cache
hits because foo.c and bar.c now will have different cache directories
and can be updated independently without clobbering each other's cache
data.
* update to the new cache hash API
* std.Target defaultVersionRange moves to std.Target.Os.Tag
* std.Target.Os gains getVersionRange which returns a tagged union
* start the process of splitting Module into Compilation and "zig
module".
- The parts of Module having to do with only compiling zig code are
extracted into ZigModule.zig.
- Next step is to rename Module to Compilation.
- After that rename ZigModule back to Module.
* implement proper cache hash usage when compiling C objects, and
properly manage the file lock of the build artifacts.
* make versions optional to match recent changes to master branch.
* proper cache hash integration for compiling zig code
* proper cache hash integration for linking even when not compiling zig
code.
* ELF LLD linking integrates with the caching system. A comment from
the source code:
Here we want to determine whether we can save time by not invoking LLD when the
output is unchanged. None of the linker options or the object files that are being
linked are in the hash that namespaces the directory we are outputting to. Therefore,
we must hash those now, and the resulting digest will form the "id" of the linking
job we are about to perform.
After a successful link, we store the id in the metadata of a symlink named "id.txt" in
the artifact directory. So, now, we check if this symlink exists, and if it matches
our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD.
* implement disable_c_depfile option
* add tracy to a few more functions
into smaller exposed components and expose all of them. This makes it
more flexible.
`*const Cache` is now passed in with an open manifest dir handle which
the caller is responsible for managing.
Expose some of the base64 stuff.
Extract the hash helper functions into `HashHelper` and add some more
methods such as addOptional and addListOfFiles.
Add `CacheHash.toOwnedLock` so that you can deinitialize everything
except the open file handle which represents the file system lock on the
build artifacts.
Use ArrayListUnmanaged, saving space per allocated CacheHash.
Avoid 1 memory allocation in hit() with a static buffer.
hit() returns a bool; caller code is responsible for calling final() in
either case. This is a simpler and easier to use API.
writeManifest() is no longer called from deinit() with errors ignored.
Speed up a little the slicing-by-8 code path by replacing the
(load+shift+xor)*4 sequence with a single u32 load plus a xor.
Before:
```
iterative: 1018 MiB/s [000000006c3b110d]
small keys: 1075 MiB/s [0035bf3dcac00000]
```
After:
```
iterative: 1114 MiB/s [000000006c3b110d]
small keys: 1324 MiB/s [0035bf3dcac00000]
```
It turns out that the kernel won't read or write more than 0x7fffffff
bytes in a single call, failing with EINVAL when trying to do so.
Adjust the limit and curse whoever is responsible for this.
Closes#6332
Move block definitions inside while loop.
Use usize for offset. (This still crashes on overflow)
Remove unneeded slice syntax.
Add slow test for Very large dkLen
Master branch added in the concept of library versioning being optional
to main.cpp. It will need to be re-added into this branch before merging
back into master.
* implement --debug-cc and --debug-link
* implement C source files having extra flags
- TODO a way to pass them on the CLI
* introduce the Directory abstraction which contains both an open file
descriptor and a file path name. The former is preferred but the
latter is needed when communicating paths over a command line (e.g.
to Clang or LLD).
* use the cache hash to choose an artifact directory
- TODO: use separate cache hash instances for the zig module and
each C object
* Module: introduce the crt_files table for keeping track of built libc
artifacts for linking.
* Add the ability to build 4/6 of the glibc static CRT lib files.
* The zig-cache directory is now passed as a parameter to Module.
* Implement the CLI logic of -femit-bin and -femit-h
- TODO: respect -fno-emit-bin
- TODO: the emit .h feature
* Add the -fvalgrind, -fstack-check, and --single-threaded CLI options.
* Implement the logic for auto detecting whether to enable PIC,
sanitize-C, stack-check, valgrind, and single-threaded.
* Properly add PIC args (or not) to clang argv.
* Implement renaming clang-compiled object files into their proper
place within the cache artifact directory.
- TODO: std lib needs a proper higher level abstraction for
std.os.renameat.
* Package is cleaned up to use the "Unmanaged" StringHashMap and use the
new Directory abstraction.
* Clean up zig lib directory detection to make proper use of directory
handles.
* Linker code invokes LLD.
- TODO properly deal with the stdout and stderr that we get from it
and expose diagnostics from the Module API that match the expected
error message format.
* Delete the bitrotted LLVM C ABI bindings. We'll resurrect just the
functions we need as we introduce dependencies on them. So far it
only has ZigLLDLink in it.
* Remove dead timer code.
* `zig env` now prints the path to the zig executable as well.
A profiler run showed that the main bottleneck was the naive decoding of
the Huffman codes, replacing it with a nice trick borrowed by Zlib gave
a substantial speedup.
Replacing a `%` with a `and (mask-1)` gave another significant
improvement (yay for low hanging fruits).
A few numbers obtained by decompressing a 22M file:
Before:
```
./decompress 2,39s user 0,00s system 99% cpu 2,400 total
```
After:
```
./decompress 0,79s user 0,00s system 99% cpu 0,798 total
````
This commit changes the behavior of stage1 to emit libfoo.so instead
of libfoo.so.0.0.0 when none of the --ver-major, --ver-minor, or
--ver-patch flags are set.
It also makes it possible to create unversioned shared libraries
using the zig build system, changing the version parameter of
addSharedLibrary() to a tagged union.
std.event.Loop does not yet work in single threaded builds. However,
using evented io on a single thread can be very convenient. This commit
allows settind @import("root").event_loop_mode to .single_threaded
in order to allow this without reimplementing the startup code in
start.zig
* add `zig libc` command
* add `--libc` CLI and integrate it with Module and linker code
* implement libc detection and paths resolution
* port LLD ELF linker line construction to stage2
* integrate dynamic linker option into Module and linker code
* implement default link_mode detection and error handling if
user requests static when it cannot be fulfilled
* integrate more linker options
* implement detection of .so.X.Y.Z file extension as a shared object
file. nice try, you can't fool me.
* correct usage text for -dynamic and -static
For when linking with LLD, we always create an object rather than going
straight to the executable. Next step is putting this object on the LLD
linker line.
* add target_util.zig which has ported code from src/target.cpp
* Module gains an arena that owns memory used during initialization
that has the same lifetime as the Module. Useful for constructing
file paths and lists of strings that have mixed lifetimes.
- The Module memory itself is allocated in this arena. init/deinit
are modified to be create/destroy.
- root_name moves to the arena and no longer needs manual free
* implement the ability to invoke `zig clang` as a subprocess
- there are lots of TODOs that should be solved before merging
* Module now requires a Random object and zig_lib_dir
* Module now requires a path to its own executable or any zig
executable that can do `zig clang`.
* Wire up more CLI options.
* Module creates "zig-cache" directory and "tmp" and "o" subdirectories
("h" is created by the cache_hash)
* stubbed out some of the things linker code needs to do with TODO
prints
* delete dead code for computing compiler id. the previous commit
eliminated the need for it.
* add `zig translate-c` CLI option but it's not fully hooked up yet.
It should be possible for this to be fully wired up before merging
this branch.
* `zig targets` now uses canonical data for available_libcs
Instead, append a "dirty suffix" to the version string when there are
dirty git changes and use the version string as the compiler id.
This avoids a dependency on the cache hash system, and saves time on
first invocation of the compiler since it does not have to compute its
compiler id. It also saves time by not having to check the cache for a
saved compiler id.
* std.cache_hash exposes Hasher type
* std.cache_hash makes hasher_init a global const
* std.cache_hash supports cloning so that clones can share the same
open manifest dir handle as well as fork from shared hasher state
* start to populate the cache_hash for stage2 builds
* remove a footgun from std.cache_hash add function
* get rid of std.Target.ObjectFormat.unknown
* rework stage2 logic for resolving output artifact names by adding
object_format as an optional parameter to std.zig.binNameAlloc
* support -Denable-llvm in stage2 tests
* Module supports the use case when there are no .zig files
* introduce c_object_table and failed_c_objects to Module
* propagate many new kinds of data from CLI into Module and into
linker.Options
* introduce -fLLVM, -fLLD, -fClang and their -fno- counterparts.
closes#6251.
- add logic for choosing when to use LLD or zig's self-hosted linker
* stub code for implementing invoking Clang to build C objects
* add -femit-h, -femit-h=foo, and -fno-emit-h CLI options
* build.zig: repair the ability to link against llvm, clang, and lld
* move the zig cc arg parsing logic to stage2
- the preprocessor flag is still TODO
- the clang arg iterator code is improved to use slices instead of
raw pointers because it no longer has to deal with an extern
struct.
* clean up error printing with a `fatal` function and use log API
for messages rather than std.debug.print
* add support for more CLI options to stage2 & update usage text
- hooking up most of these new options is TODO
* clean up the way libc and libc++ are detected via command line
options. target information is used to determine if any of the libc
candidate names are chosen.
* add native library directory detection
* implement the ability to invoke clang from stage2
* introduce a build_options.have_llvm so we can comptime branch
on whether LLVM is linked in or not.
It seems MachO does not like padding between text block in __text
section. Unlike in Elf, there is no size information in symbol
struct `nlist_64`.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
Fixes a bug where the last written load command would accidentally
override the beginning of the __text section. Also defines missing
MachO constants and relocation structs/enums.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
Add check if the type is float. Float byte comparison doesn't work
because +0.0 and -0.0 are considered equal but their byte
representations are not equal.
* Add a size_hint parameter to the read{toEnd,File}AllocOptions fns
* Rename readAllAlloc{,Options} to readToEndAlloc{,Options} as they
don't rewind the file before reading
* Fix missing rewind in test case
Added std.coff.MachineType
Added image characteristic and section flag valued to std.coff
Added std.Target.Cpu.Arch.toCoffMachine
Fixed stage2 --watch flag on windows
This commit adds enough Mach-O linker implementation to write out simple
Mach-O object file. Be warned however, the object file is largely incomplete:
misses relocation info, debug symbols, etc. However, it seemed like a
good starting to get the basic understanding right.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
- correct uid_t from i32 to u32 on linux
- define uid_t and gid_t for OSes missing definitions
- use uid_t/gid_t instead of plain u32s throughout std.os
Don't use the user-supplied cap as starting point for a resize. Doing so
overallocates memory and thus negates the whole point of specifying a
precise cap value.
* improve the ZIR generated of variable decls
- utilize the same ZIR for the type and init value when possible
- init value gets a result location with the variable type.
no manual coercion is required.
* no longer use return instructions to extract values out of comptime
blocks. Instead run the analysis and then look at the corresponding
analyzed instruction, relying on the comptime mechanism to report
errors when something could not be comptime evaluated.
We're up against the 6 hour limit so this is a time-saving workaround.
Compile error tests are generally not OS-specific so the coverage from
the other platforms should be sufficient.
When upgrading to the new std lib HashMap API, the process_headers code
regressed because something that was supposed to be a pointer ended up
being a copy of a value. This resulted in the modification of a field
not being picked up.
Also switch from Sha256 to Blake3 while we're at it.
The backing allocator may return a block that's actually bigger than the
one required by the user, use the correct quantity when keeping track of
the allocation ceiling.
Closes#6049
- Use an enum of all field names instead of string literals
- Create a struct type with all fields optional instead of relying on
anonymous struct literals
This should provide better type inference, compile errors, and a
(subjectively) cleaner API.
Avoids a compile error from start.zig:
/home/kivikakk/zig/build/lib/zig/std/start.zig:265:28: error:
expected type 'u8', found 'i8'
return result;
^
/home/kivikakk/zig/build/lib/zig/std/start.zig:265:28: note:
unsigned 8-bit int cannot represent all possible signed 8-bit
values
return result;
- 1MiB objects on the stack doesn't play well with wasmtime.
Reduce these to 512KiB so that the webassembly benchmarks can run.
- Pass expected results to a blackBox() function. Without this, in
release-fast mode, the compiler could detected unused return values,
and would produce results that didn't make sense for siphash.
- Add AEAD constructions to the benchmarks.
- Inline chacha20Core() makes it 4 times faster.
- benchmarkSignatures() -> benchmarkSignature() for consistency.
* Move branch-local register and stack allocation metadata to the
function-local struct. Conditional branches clone this data in order
to restore it after generating machine code for a branch.
Branch-local data is now only the instruction table mapping *ir.Inst
to MCValue.
* Implement conditional branching
- Process operand deaths
- Handle register and stack allocation metadata
* Avoid storing unreferenced or void typed instructions into
the branch-local instruction table.
* Fix integer types reporting the wrong value for hasCodeGenBits.
* Remove the codegen optimization for eliding length-0 jumps. I need to
reexamine how this works because it was causing invalid jumps to be
emitted.
While we try to work out what the correlation between the OS and runtime
versions is, this commit hardcodes the latter to the minimum (compat)
version of 1.0.0.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
These changes enable a Hello World example. However, all implemented
codegen is not yet feature-complete.
- asm only supports 'svc #0' at the moment
- call only supports leaf functions at the moment
- setReg uses a naive method at the moment
* move SPU code from std to self hosted compiler
* change std lib comments to be descriptive rather than prescriptive
* avoid usingnamespace
* fix case style of error codes
* remove duplication of producer_string
* generalize handling of less than 64 bit arch pointers
* clean up SPU II related test harness code
I forgot to do -Denable-qemu -Denable-wasmtime when testing yesterday,
sorry about that.
In reuseOperand, the code assumed a re-used register would be tracked in
the register table but that is not always the case.
This makes the `cache_hash` hash function easier to replace.
BLAKE3 would be a natural fit for hashing large files, but:
- second preimage resistance is not necessary for the cache_hash use cases
- our BLAKE3 implementation is currently very slow
Switch to SipHash128, which gives us an immediate speed boost.
* moved bpf syscall, added some bpf instructions and tests
* had to move bpf out of bits so that a freestanding target could import it
* removed line
* fixed imports
The `entry_point_command` is a replacement for `thread_command`, and
is used for main executables to specify the location of `main()`
entry point.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
SipHash *is* a cryptographic function, with a 128-bit security level.
However, it is not a regular hash function: a secret key is required,
and knowledge of that key allows collisions to be quickly computed offline.
SipHash is therefore more suitable to be used as a MAC.
The same API as other MACs was implemented in addition to functions directly
returning an integer.
The benchmarks have been updated accordingly.
No changes to the SipHash implementation itself.
See #6113 for an alternate way of doing this that we didn't end up
following.
Closes#6079.
I also took the opportunity here to extract C.zig and Elf.zig from
link.zig.
The API of SinglyLinkedList was changed in 93384f7, removing the init
function as well as the redundant allocation helper functions.
This commit makes parallel changes to the API of TailQueue in order to
keep the standard library consistent.
- This avoids having multiple `init()` functions for every combination
of optional parameters
- The API is consistent across all hash functions
- New options can be added later without breaking existing applications.
For example, this is going to come in handy if we implement parallelization
for BLAKE2 and BLAKE3.
- We don't have a mix of snake_case and camelCase functions any more, at
least in the public crypto API
Support for BLAKE2 salt and personalization (more commonly called context)
parameters have been implemented by the way to illustrate this.
Justification:
- reset() is unnecessary; states that have to be reused can be copied
- reset() is error-prone. Copying a previous state prevents forgetting
struct members.
- reset() forces implementation to store sensitive data (key, initial state)
in memory even when they are not needed.
- reset() is confusing as it has a different meaning elsewhere in Zig.
Instead of having all primitives and constructions share the same namespace,
they are now organized by category and function family.
Types within the same category are expected to share the exact same API.
According to the Mach-O file format reference, the first
load command should be a `__PAGEZERO` segment command. The
segment is located at virtual memory location 0, has no protection
rights, and causes acccesses to NULL to immediately crash.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
`is_pub` added to `Fn` would cost us an additional 8
bytes of memory per function, which is a real bummer
since it's only 1 bit of information.
If we wanted to really remove this, I suspect we could
make this a function isPub() which looks at the AST of
the corresponding Decl and finds if the FnProto AST node
has the pub token. However I saw an easier approach -
The data of whether something is pub or not is actually
a property of a Decl anyway, not a function, so we can
look at moving the field into Decl. Indeed, doing this,
we see that Decl already has deletion_flag: bool which
is hiding in the padding bytes between the enum (1 byte)
and the following u32 field (generation). So if we put
the is_pub bool there, it actually will take up no
additional space, with 1 byte of padding remaining.
This was an easy reworking of the code since any
func.is_pub could be changed simply to func.owner_decl.is_pub.
I also modified `Var` to make the init value non-optional
and moved the optional bit to a has_init: bool field. This is worse from
the perspective of control flow and safety, however it makes
`@sizeOf(Var)` go from 32 bytes to 24 bytes. The more code we can fit
into memory at once, the more justified we are in using the compiler as
a long-running process that does incremental updates.
During codegen we do not yet know the indexes that will be used for
called functions. Therefore, we store the offset into the in-memory
code where the index is needed with a pointer to the Decl and use this
data to insert the proper indexes while writing the binary in the flush
function.
Before this commit the wasm backend worked similarly to elf. As
functions were generated they were written directly to the output file
and existing code was shifted around in the file as necessary. This
approach had several disadvantages:
- Large amounts of padding in the output were necessary to avoid
expensive copying of data within the file.
- Function/type/global/etc indexes were required to be known at the time
of preforming codegen, which severely limited the flexibility of where
code could be placed in the binary
- Significant complexity to track the state of the output file through
incremental updates
This commit takes things in a different direction. Code is incrementally
compiled into in-memory buffers and the entire binary is rewritten using
these buffers on flush. This has several advantages:
- Significantly smaller resulting binaries
- More performant resulting binaries due to lack of indirection
- Significantly simpler compiler code
- Indexes no longer need to be known before codegen. We can track where
Decls must be referenced by index insert the proper indexes while
writing the code in the flush() function. This is not yet implemented
but is planned for the next commit.
The main disadvantage is of course increased memory usage in order to
store these buffers of generated code.
This is a temporary debugging trick you can use to turn segfaults into more helpful
logged error messages with stack trace details. The downside is that every allocation
will be leaked!
This commit write out Mach-O header in the linker's `flush`
method. The header currently only populates the magic number,
filetype, and cpu info.
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
I have observed on Linux writing and reading the same file many times
without the mtime changing, despite the file system having nanosecond
granularity (and about 1 millisecond worth of nanoseconds passing between
modifications). I am calling this a Linux Kernel Bug and adding file
size to the cache hash manifest as a mitigation. As evidence, macOS does
not exhibit this behavior.
This means it is possible, on Linux, for a file to be added to the cache
hash, and, if it is updated with the same file size, same inode, within
about 1 millisecond, the cache system will give us a false positive,
saying it is unmodified. I don't see any way to improve this situation
without fixing the bug in the Linux kernel.
closes#6082
* Implemented all R-type arithmetic/logical instructions
* Implemented all I-type arithmetic/logical instructions
* Implemented all load and store instructions
* Implemented all of RV64I except FENCE
I empirically observed mtime not changing when rapidly writing the same
file name within the same millisecond of wall clock time, despite the
mtime field having nanosecond precision.
I believe this fixes the CI test failures.
Functions which are free'd are not immediately removed from the binary
as this would cause a shifting of function indexes. Instead, they hang
around until they can be overwritten by a new function. This means that
the types associated with these dead functions must also remain until
the function is overwritten to avoid a type mismatch.
Exports now have a dirty flag and are rewritten on flush if this flag
has been set.
A couple other minor changes have been made based on Andrew's review.
Thus far, we only generate the type, function, export, and code
sections. These are sufficient to generate and export simple functions.
Codegen is currently hardcoded to `i32.const 42`, the main goal of this
commit is to create infrastructure for the container format which will
work with incremental compilation.
* Factor redundant code in std/crypto/chacha20
* Add support for XChaCha20, and the XChaCha20-Poly1305 construction.
XChaCha20 is a 24-byte version of ChaCha20, is widely implemented
and is on the standards track:
https://tools.ietf.org/html/draft-irtf-cfrg-xchacha-03
* Add support for encryption/decryption with the authentication tag
detached from the ciphertext
* Add wrappers with an API similar to the Gimli AEAD type, so that
we can use and benchmark AEADs with a common API.
This is a rewrite of the x25519 code, that generalizes support for
common primitives based on the same finite field.
- Low-level operations can now be performed over the curve25519 and
edwards25519 curves, as well as the ristretto255 group.
- Ed25519 signatures have been implemented.
- X25519 is now about twice as fast.
- mem.timingSafeEqual() has been added for constant-time comparison.
Domains have been clearly separated, making it easier to later add
platform-specific implementations.
We still want this compile error but I'm giving up on implementing it
correctly in stage1. It's been buggy and has false positives sometimes.
I left the test cases there, but commented out, so that when we go
through the stage1 compile error cases and get coverage for them in
stage2 we can reactivate the test cases.
closes#2154
This is part of an ongoing effort to reduce size of in-memory AST. This
enum flattening pattern is widespread throughout the self-hosted
compiler.
This is a API breaking change for consumers of the self-hosted parser.
* introduce a dump() function on Module.Fn which helpfully prints to
stderr the ZIR representation of a function (can be called before
attempting to codegen it). This is a debugging tool.
* implement x86 codegen for loops
* liveness: fix analysis of conditional branches. The logic was buggy
in a couple ways:
- it never actually saved the results into the IR instruction (fixed now)
- it incorrectly labeled operands as dying when their true death was
after the conditional branch ended (fixed now)
* zir rendering is enhanced to show liveness analysis results. this
helps when debugging liveness analysis.
* fix bug in zir rendering not numbering instructions correctly
closes#6021
It wasn't wrong info, but e.g. GDB couldn't handle non-power-of-two
enum tags. Now we tell debug info that enum tags are always power of two
size.
closes#4526closes#5432
The logging functions in std.log don't take a scope parameter anymore,
but use the .default scope. To provide your own scope, use the logging
functions in std.log.scoped(.some_other_scope).
As per nmichaels' suggestion: https://github.com/ziglang/zig/pull/6039#issuecomment-673148971
`std.os.getFdPath` is very platform-specific and can be used to query
the OS for a canonical path to a file handle. Currently supported hosts
are Linux, macOS and Windows.
`std.fs.Dir.realpath` (and null-terminated, plus WTF16 versions) are
similar to `std.os.realpath`, however, they resolve a path wrt to this
`Dir` instance.
If the input pathname argument turns out to be an absolute path, this
function reverts to calling `realpath` on that pathname completely
ignoring this `Dir`.
* Add short documentation to std.log.scoped and std.log.default
* Update the module documentation and example to explain the difference
between using explicit scopes, using a scoped logging namespace, and
using the default namespace
* Add a std.log.scoped function that returns a scoped logging struct
* Add a std.log.default struct that logs using the .default scope
Implementation of daurnimator's proposal:
https://github.com/ziglang/zig/issues/5943#issuecomment-669043489
Note that I named the function "scoped" instead of "scope" so as not to
clash with the scope parameter that is used everywhere; this seemed a
better solution to me than renaming the scope parameter to "s" or
"log_scope" or the like.
This makes collected stack traces omit less useful frames. For user
applications which only store a fixed number of stack frames this can
make a big difference.
`std.builtin.StackTrace` gains a `format` function.
GeneralPurposeAllocator uses `std.log.err` instead of directly printing
to stderr. Some errors are recoverable.
The test runner is modified to fail the test run if any log messages of
"err" or worse severity are encountered.
self-hosted is modified to always print log messages of "err" severity
or worse even if they have not been explicitly enabled.
This makes GeneralPurposeAllocator available on the freestanding target.
We don't pass no-omit-frame-pointer in release safe by default, so it
also makes sense to not try to collect stack trace frames by default in
release safe mode.
This makes `@returnAddress()` return 0 for WebAssembly (when not using
the Emscripten OS) and avoids trying to capture stack traces for the
general purpose allocator on that target.
The high level Allocator interface API functions will now do a
`@returnAddress()` so that stack traces captured by allocator
implementations have a return address that does not include the
Allocator overhead functions. This makes `4` a more reasonable default
for how many stack frames to capture.
* std.Mutex API is improved to not have init() deinit(). This API is
designed to support static initialization and does not require any
resource cleanup. This also happens to work around some kind of
stage1 behavior that wasn't letting the new allocator mutex code
get compiled.
* the general purpose allocator now returns a bool from deinit()
which tells if there were any leaks. This value is used by the test
runner to fail the tests if there are any.
* self-hosted compiler is updated to use the general purpose allocator
when not linking against libc.
`std.GeneralPurposeAllocator` is now available. It is a function that
takes a configuration struct (with default field values) and returns an
allocator. There is a detailed description of this allocator in the
doc comments at the top of the new file.
The main feature of this allocator is that it is *safe*. It
prevents double-free, use-after-free, and detects leaks.
Some deprecation compile errors are removed.
The Allocator interface gains `old_align` as a new parameter to
`resizeFn`. This is useful to quickly look up allocations.
`std.heap.page_allocator` is improved to use mmap address hints to avoid
obtaining the same virtual address pages when unmapping and mapping
pages. The new general purpose allocator uses the page allocator as its
backing allocator by default.
`std.testing.allocator` is replaced with usage of this new allocator,
which does leak checking, and so the LeakCheckAllocator is retired.
stage1 is improved so that the `@typeInfo` of a pointer has a lazy value
for the alignment of the child type, to avoid false dependency loops
when dealing with pointers to async function frames.
The `std.mem.Allocator` interface is refactored to be in its own file.
`std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`.
This allocator is great for debug mode, however it needs some work to
have better performance in release modes. The next step will be setting
up a series of tests in ziglang/gotta-go-fast and then making
improvements to the implementation.
This commit reimagines `std.os.windows.GetFinalPathNameByHandle`
using `DeviceIoControl` to query the OS mount manager for the DOS
(symlink) paths for the given NT volume name. In particular,
it uses `IOCTL_MOUNTMGR_QUERY_POINTS` ioctl opcode to query the
manager for the available moount points.
Favour newer API which uses `NtQueryInformationFile` with class flags
`FileNormalizedNameInformation` and `FileVolumeNameInformation`
instead of lower-level `NtQueryObject`. `NtQueryObject` is still
used as a fallback in case the former are unavailable.
This commit proposes an initial draft of `GetPathNameByHandle` function
which wraps NT syscalls and strives to emulate (currently only
partially) the `kernel32.GetFinalPathNameByHandleW` function.
I made two mistakes in the previous commit; it was not actually using
the argv that we built, and also the qemu logic was unconditionally
skipping the test.
Now I have verified that when mangling the RISC-V "hello world" test and
then using -Denable-qemu, we get a test failure.
SrcFn represents the function in the linked output file, if the
`Decl` is a function. This is stored here and not in `Fn` because `Decl`
survives across updates but `Fn` does not.
TODO Look into making `Fn` a longer lived structure and moving this field there
to save on memory usage.
Empirically, debug info consumers do not respect this field, or otherwise
consider it to be an error when it does not point exactly to the end of the header.
Therefore we rely on the NOP jump at the beginning of the Line Number Program for
padding rather than this field.
llvm-dwarfdump says the line number data is fine; gdb and
binutils-readelf crap out.
Supports writing the first function. Still TODO is:
* handling the .debug_line header growing too large
* adding a new file to an existing compilation
* adding an additional function to an existing file
* handling incremental updates
* adding the main IR debug ops for IR instructions
There are also issues to work out:
* readelf --debug-dump=rawline is saying there is no .debug_str section
even though there is
* readelf --debug-dump=decodedline is saying the file index 0 is bad
and reporting some other kind of corruption.
* the .debug_line header is written properly
* link.File.Elf gains:
- SrcFn, which is now a field in Module.Fn
- SrcFile, which is now a field in Module.Scope.File
* link.File.Elf gets a whole *Package field rather than only
root_src_dir_path.
* the fields first_dbg_line_file and last_dbg_line_file tell where the
Line Number Program begins and ends, which alows moving files when
the header gets too big, and allows appending files to the end.
* codegen is passed a buffer for emitting .debug_line
Line Number Program opcodes for functions.
See #5963
There is some work-in-progress code here, but I need to go make some
experimental changes to changing how to represent source locations and I
want to do that in a separate commit.
This shaves off one syscall (we use one instead of two if we were to
use `windows.OpenFile` wrapper). Clean up flag generation in `OpenFile`.
Hopefully, we're in a much better place to *almost* support `openW`
and `openatW`.
This way, we can remove more `kernel32` calls such as `RemoveDirectoryW`
or `DeleteFileW`, and use `std.os.windows.DeleteFile` instead which
is purely NT-based.
Replace them with `std.os.windows.OpenFile` instead. To allow
creation/opening of directories, `std.os.windows.OpenFileOptions`
now features a `.expect_dir: bool` member which is meant to emualate
POSIX's `O_DIRECTORY` flag.
As discussed in the previous commit, it would be better to avoid
function pointers to syscalls and explicitly split the control
path into two function calls instead. This commit addresses that
for `std.os.windows.DeviceIoControl`.
* multiple returns jump to one canonical function exitlude. This is in
preparation for the defer feature.
* simple elision of trivial jump relocs.
* omit prelude/exitlude for naked calling convention functions.
* fix not switching on arch for prelude/exitlude
* fix swapped registers when setting stack mem from a register
* `optimize_mode` is passed to `link.File` and stored there
* improve the debugging function `Module.dumpInst`
* get rid of `Value.the_one_possible_value` in favor of a few more
specific values for different types. This is less buggy, one less
footgun.
* `Type.onePossibleValue` now returns a `?Value` instead of `bool`.
* codegen handles undefined values. `undef` is a new `MCValue` tag.
It uses 0xaa values depending on optimization mode. However
optimization mode does not yet support scope overrides.
* link.zig: move the `Options` field from `File.Elf` and `File.C` to
the base struct.
- fix the Tag enum to adhere to style conventions
* ZIR now supports emitting undefined values.
* Fix the logic of comptime math to properly compare against zero using
the `compareWithZero` function.
* implement sema for runtime deref, store pointer, coerce_to_ptr_elem,
and store
* identifiers support being lvalues, except for decls is still TODO
* codegen supports load, store, ref, alloc
* introduce more MCValue union tags to support pointers
* add load, ref, store typed IR instructions
* add Type.isVolatilePtr
When there are no format parameters, it simply calls `writeAll`. This
has the effect of no longer emitting a compile error for using `{}` and
not having any parameters, however, at this point in the development
process of Zig I think that tradeoff is worthwhile.
On the other hand, it might be OK to simply define formatting to work
this way. It's a common pattern to use the formatting function's format
string `"like this", .{}` instead of `"{}", .{"like this"}`, which can
lead to accidentally putting control characters in the formatting
string, however, with this change that works just fine.
This commit adds a Zig wrapper for `kernel32.DeviceIoControl` which
applies ReactOS logic for deciding whether to use
`ntdll.NtDeviceIoControlFile` or `ntdll.NtFsControlFile` based on the
value of passed `IO_CONTROL_CODE`. The decision logic is based on the
logic found in ReactOS found in the following [link].
Thanks to Daurnimator for finding this bit in ReactOS!
[link]: https://doxygen.reactos.org/d3/d74/deviceio_8c.html
Comment out non-x86_64 architectures for now in codegen.zig, because
they all have compile errors for their codepaths anyway, and it was
bloating the compilation speed and memory usage when stage1 tried to
build self-hosted. Here's the panic message:
"Backend architectures that don't have good support yet are commented
out, to improve compilation performance. If you are interested in one
of these other backends feel free to uncomment them. Eventually these
will be completed, but stage1 is slow and a memory hog."
This is a workaround to lower the time it takes to build self-hosted
with stage1 as well as use less memory. It should fix the CI.
Additionally:
* Add `single_mut_pointer` support to `Type`
* Trivial implementation of stack allocation in codegen.zig. It does
not deal with freeing yet, and it's missing the stack pointer
adjustment prologue.
* Add the `alloc` IR instruction and semantic analysis for `alloc` ZIR
instruction.
This is an attempt to save memory when building self-hosted.
before:
Total bytes allocated: 5.941 GiB, deallocated: 2.259 GiB, remaining: 3.681 GiB
after:
Total bytes allocated: 5.933 GiB, deallocated: 2.253 GiB, remaining: 3.680 GiB
This makes sense from an organizational point of view, as explained by
this new doc comment at the top of the new file:
//! Semantic analysis of ZIR instructions.
//! This file operates on a `Module` instance, transforming untyped ZIR
//! instructions into semantically-analyzed IR instructions. It does type
//! checking, comptime control flow, and safety-check generation. This is the
//! the heart of the Zig compiler.
//! When deciding if something goes into this file or into Module, here is a
//! guiding principle: if it has to do with (untyped) ZIR instructions, it goes
//! here. If the analysis operates on typed IR instructions, it goes in Module.
Before:
4009 src-self-hosted/Module.zig
After:
2776 src-self-hosted/Module.zig
1128 src-self-hosted/zir_sema.zig
This should be sufficient to avoid the situation we have in stage1 where
ir.cpp is 32,516 lines.
Previously, `unsigned` was parsed as the shorthand for `unsigned int`.
This commit introduces code to parse `unsigned short`, `unsigned int`,
`unsigned long`, and `unsigned long long`.
There is a comment in the code about std.c.parse` - Im not
familiar with zig internals, but it seems like this is a separate
C parsing implementation. In the long run, it probably makes
sense to merge both implementations, so this commit should be
regarded as a quick fix that doesn't address an apparently
underlying issue.
* Added `replace` for simple pre-allocated search-and-replace
* Added `replacementSize` for calculating ahead-of-time buffer sizes for
performing a safe search-and-replace
* Added `replaceOwned` for automatically allocating and performing
replacement
This commit makes it possible to obtain pointers to `extern` variables
at comptime.
- `ir_get_var_ptr` employs several checks to determine if the given
variable is eligible for obtaining its pointer at comptime. This
commit alters these checks to consider `extern` variables, which have
runtime values, as eligible.
- After this change, it's now possible for `render_const_val` to be
called for `extern` variables. This commit modifies
`render_const_val` to suppress the value generation for `extern`
variables.
- `do_code_gen` now creates `ZigValue::llvm_global` of `extern`
variables before iterating through module-level variables so that
other module-level variables can refer to them.
This solution is incomplete since there are several cases still
failing:
- `global_var.array[n..m]`
- `&global_var.array[i]`
- `&global_var.inner_struct.value`
- `&global_array[i]`
Closes#5349
* bring `construct_linker_job_macho` to parity with
`construct_linker_job_elf`
* macho now sets `-error-limit`
* macho on macOS now sets `-macosx_version_min` and `-sdk_version`
to `10.13` when running `zig0`
* macho now detects when `-l` prefix is not needed
* macho on macOS detects system libraries in a case-insensitive manner
* macho now ads user-specified libraries to linker command-line args
when condition `is_native_os != true`
* re-ordered some macho args positions to match elf positions
closes#5059closes#5067
* AST: flatten ControlFlowExpression into Continue, Break, and Return.
* AST: unify identifiers and literals into the same AST type: OneToken
* AST: ControlFlowExpression uses TrailerFlags to optimize storage
space.
* astgen: support `var` as well as `const` locals, and support
explicitly typed locals. Corresponding Module and codegen code is not
implemented yet.
* astgen: support result locations.
* ZIR: add the following instructions (see the corresponding doc
comments for explanations of semantics):
- alloc
- alloc_inferred
- bitcast_result_ptr
- coerce_result_block_ptr
- coerce_result_ptr
- coerce_to_ptr_elem
- ensure_result_used
- ensure_result_non_error
- ret_ptr
- ret_type
- store
- param_type
* the skeleton structure for result locations is set up. It's looking
pretty clean so far.
* add compile error for unused result and compile error for discarding
errors.
* astgen: split builtin calls up to implemented manually, and implement
`@as`, `@bitCast` (and others) with respect to result locations.
* add CLI support for hex and raw object formats. They are not
supported by the self-hosted compiler yet, and emit errors.
* rename `--c` CLI to `-ofmt=[objectformat]` which can be any of the
object formats. Only ELF and C are supported so far. Also added missing
help to the help text.
* Remove hard tabs from C backend test cases. Shame on you Noam, you
are grounded, you should know better, etc. Bad boy.
* Delete C backend code and test case that relied on comptime_int
incorrectly making it all the way to codegen.
Otherwise, the behaviour can lead to unexpected results, resulting
in removing an entire tree that's not necessarily under the root.
Furthermore, this change is needed if are to properly handle dir
symlinks on Windows. Without explicitly requiring that a directory
or file is opened with `FILE_OPEN_REPARSE_POINT`, Windows automatically
dereferences all symlinks along the way. This commit adds another
option to `OpenDirOptions`, namely `.no_follow`, which defaults to
`false` and can be used to specifically open a directory symlink on
Windows or call `openat` with `O_NOFOLLOW` flag in POSIX.
This way `std.fs.symlinkAbsolute` becomes cross-platform and we can
legally include `SymlinkFlags` as an argument that's only used on
Windows. Also, now `std.os.symlink` generates a compile error on
Windows with a message to instead use `std.os.windows.CreateSymbolicLink`.
Finally, this PR also reshuffles the tests between `std.os.test` and
`std.fs.test`.
These are now supported enough that this example code hits the
limitations of the register allocator:
fn add(a: u32, b: u32) void {
const c = a + b; // 7
const d = a + c; // 10
const e = d + b; // 14
assert(e == 14);
}
// error: TODO implement copyToNewRegister
So now the next step is to implement register allocation as planned.
InfixOp is flattened out so that each operator is an independent AST
node tag. The two kinds of structs are now Catch and SimpleInfixOp.
Beginning implementation of supporting codegen for const locals.
Now that Big Sur does not have system libraries on the filesystem, zig can no longer read them and add them to the cache hash for the compiler id.
This changes it so that only the first library path returned by os_self_exe_shared_libs is added to the cache hash under Darwin. I looked into methods on getting the system version to keep parity with older versions, but @fengb reported that this works on Catalina (a version behind Big Sur)
Signed-off-by: Haze Booth <isnt@haze.cool>
ast.Node.Id => ast.Node.Tag, matching recent style conventions.
Now multiple different AST node tags can map to the same AST node data
structures. In this commit, simple prefix operators now all map top
SimplePrefixOp.
`ast.Node.castTag` is now preferred over `ast.Node.cast`.
Upcoming: InfixOp flattened out.
These AST nodes now have a flags field and then a bunch of optional
trailing objects. The end result is lower memory usage and consequently
better performance. This is part of an ongoing effort to reduce the
amount of memory parsed ASTs take up.
Running `zig fmt` on the std lib:
* cache-misses: 2,554,321 => 2,534,745
* instructions: 3,293,220,119 => 3,302,479,874
* peak memory: 74.0 MiB => 73.0 MiB
Holding the entire std lib AST in memory at the same time:
93.9 MiB => 88.5 MiB
This is useful for saving memory when allocating an object that has many
optional components. The optional objects are allocated sequentially in
memory, and a single integer is used to represent each optional object
and whether it is present based on each corresponding bit.
This is part of a larger effort to improve the memory layout of AST
nodes of the self-hosted parser to reduce wasted memory. Reduction of
wasted memory also translates to improved performance because of fewer
memory allocations, and fewer cache misses.
Compared to master, when running `zig fmt` on the std lib:
* cache-misses: 801,829 => 768,624
* instructions: 3,234,877,167 => 3,232,075,022
* peak memory: 81480 KB => 75964 KB
Also, check for overflow on incremented file descriptors. Previously,
we'd trigger a panic if we exceeded the `fd_t` resolution. Now, instead,
we throw an `error.Overflow` to signal that there can be no more
file descriptors available from the runtime. This way we give the user
the ability to still be able to check if their desired preopen exists
in the list or not.
This commit edits the "Hello, World!" introduction. It introduces Error Union
Types. Also, it changes `outStream` to `writer` in the code example and description.
To introduce the Zig programming language, the "Hello, world!" code sample now has
documentation to explain some of the features shown in the code sample
and contains links to those features in the rest of the documentation.
Writing style goals:
* Balance writing style to keep beginner and experience programmers interested.
* Be concise: allow the rest of the documentation to clarify language features.
This is an edge case that isn't too uncommon but is rather confusing to try to deduce without documentation, since it feels like `else` is being overloaded in this scenario and there's no obvious 'correct' behavior here. This just adds a test demonstrating how Zig currently behaves in this scenario.
The language reference's Index is a list of the documentation's contents in
order of appearance. This commit renames "Index" to "Contents" as in table of
contents. It also renames the HTML/CSS identifiers from "index" to "toc".
* breaking changes to the API. Some of the weird decisions from before
are changed to what would be more expected.
- `get` returns `?V`, use `getEntry` for the old API.
- `put` returns `!void`, use `fetchPut` for the old API.
* HashMap now has a comptime parameter of whether to store hashes with
entries. AutoHashMap has heuristics on whether to set this parameter.
For example, for integers, it is false, since equality checking is
cheap, but for strings, it is true, since equality checking is
probably expensive.
* The implementation has a separate array for entry_index /
distance_from_start_index. Entries no longer has holes; it is an
ArrayList, and iteration is simpler and more cache coherent.
This is inspired by Python's new dictionaries.
* HashMap is separated into an "unmanaged" and a "managed" API. The
unmanaged API is where the actual implementation is; the managed API
wraps it and provides a more convenient API, storing the allocator.
* Memory usage: When there are less than or equal to 8 entries, HashMap
now incurs only a single pointer-size integer as overhead, opposed to
using an ArrayList.
* Since the entries array is separate from the indexes array, the holes
in the indexes array take up less room than the holes in the entries
array otherwise would. However the entries array also allocates
additional capacity for appending into the array.
* HashMap now maintains insertion order. Deletion performs a "swap
remove". It's now possible to modify the HashMap while iterating.
This adds these two fields to a HashMap Entry:
uint32_t hash
uint32_t distance_from_start_index
Compared to master branch, standard library tests compiled 8.4% faster
and took negligible (0.001%) more memory to complete. The amount of
memory used is still down from before 8b82c40104 which moved indexes
to be stored separately from entries.
So, it turns out, keeping robin hood hashing plus separating indexes
did result in a performance improvement. What happened previously is
that the gains from separating indexes balanced out the losses from
removing robin hood hashing, resulting in a wash.
This also serves as an inspiration for adding a benchmark to
std.AutoHashMap and improving the implementation.
The indexes are stored separately using an array of
uint8_t, uint16_t, uint32_t, or size_t, depending on the number of
entries in the map.
Entries only contain a key and a value, no longer have
distance_from_start_index or is_used.
In theory this should be both faster and use less memory.
In practice it seems to have little to no effect. For the standard
library tests, vs master branch, the time had no discernable
difference, and it shaved off only 13 MiB of peak rss usage.
This is a bit hacky since we end up doing more than just grabbing
the stack pointer in the inline assembly block. Ideally _start would
be implemented in pure asm for powerpc64le, but this will do for now.
Still to be implemented is powerpc, powerpc64, and powerpc64 (ELFv2)
support. The latter will just require correctly determing target ABI
for powerpc64 and enabling the existing powerpc64le implementation for
it.
This is direct result of review comments left by andrewrk and
daurnimator. It makes sense to map `ENOTCAPABLE` into a more generic
`error.AccessDenied`.
This commit adds `error.NotCapable` enum value and makes sure that
every applicable WASI syscall that can return `ENOTCAPABLE` errno
remaps it to `error.NotCapable.
In codegen.zig, the std.Target.Cpu.Arch is now generally available as a
comptime value where needed. This is a tradeoff that causes the compiler
binary to be more bloated, but gives us higher performance, since the
optimizer can optimize per architecture (which is usually how compilers
are designed anyway, with different code per-architecture), and it also
allows us to use per-architecture types, such as a Register enum that is
specific to the comptime-known architecture.
Adds abiSize method to Type.
Linux deviates from POSIX and returns EISDIR while other POSIX systems return EPERM. To make all platforms consistent in their errors when calling deleteFile on a directory, we have to do a stat to translate EPERM (AccessDenied) to EISDIR (IsDir).
I'm allowing incremental compilation of ZIR modules to be broken. This
is not a real use case of ZIR, and the feature requires a lot of code
duplication with incremental compilation of Zig AST (which works great).
Also, add more informative `@compileError` in a few `std.os` functions
that would otherwise yield a cryptic compile error when targeting
WASI. Finally, enhance docs in a few places and add test case for
`fstatat`.
This reverts commit 93f0bcb649.
This is not correct (from #mingw-w64 IRC):
<andrewrk> is there no .def file for ntdll on arm 64 bit?
<andrewrk> or does mingw-w64-crt/lib64/ntdll.def apply to both x86_64
and aarch64?
<wbs> andrewrk: there's none at the moment (as apps rarely link directly
against ntdll, and I didn't want to guess needlessly around that one
originally when I added arm64 support, before I actually had a real
arm64 windows device)
<wbs> andrewrk: but I guess I could/should complete that now
<wbs> (if you need one right now, the libarm32 one probably is the
closest match)
* anonymous decls have automatically generated names and symbols, and
participate in the same memory management as named decls.
* the Ref instruction is deleted
* the DeclRef instruction now takes a `[]const u8` and DeclRefStr takes
an arbitrary string instruction operand.
* introduce a `zir.Decl` type for ZIR Module decls which holds
content_hash and name - fields that are not needed for `zir.Inst`
which are created as part of semantic analysis. This improves the
function signatures of Module.zig and lowers memory usage.
* the Str instruction is now defined to create an anonymous Decl and
reference it.
This commit generalizes `std.fs.wasi.PreopenList.find(...)` allowing
search by `std.fs.wasi.PreopenType` union type rather than by dir
name. In the future releases of WASI, it is expected to have more
preopen types (or capabilities) than just directories. This commit
aligns itself with that vision.
This is a potentially breaking change. However, since `std.fs.wasi.PreopenList`
wasn't made part of any Zig release yet, I think we should be OK
to introduce those changes without pointing to any deprecations.
This commit adds some unit tests for `std.fs.File.readAllAlloc`
function. It also updates the docs of `Reader.readNoEof`
which were outdated, and swaps `inStream()` for `reader()` in
`File.readAllAlloc` with the former being deprecated.
Adds Windows stub (still needs to be implemented on Windows),
adds WASI implementation, adds unit test testing basic chain of
ops: create file -> symlink -> readlink.
zig fmt previously would write a temp file, and then either rename it
into place if necessary, or unlink it if nothing was changed. Now zig
fmt renders into a memory buffer, and only writes the temp file and
renames it into place if anything changed.
Based on the performance testing I did this actually did not have much
of an impact, however it's likely that on other operating systems and
other hard drives this could make a big difference.
* `std.fs.Dir.Entry.Kind` is moved to `std.fs.File.Kind`
* `std.fs.File.Stat` gains the `kind` field, so performing a stat() on
a File now tells what kind of file it is. On Windows this only will
distinguish between directories and files.
* rework zig fmt logic so that in the case of opening a file and
discovering it to be a directory, it closes the file descriptor
before re-opening it with O_DIRECTORY, using fewer simultaneous open
file descriptors when walking a directory tree.
* rework zig fmt logic so that it pays attention to the kind of
directory entries, and when it sees a sub-directory it attempts to
open it as a directory rather than a file, reducing the number of
open() syscalls when walking a directory tree.
The original check for a directory was for the `readAllAlloc` so move the check from open to read. This in turn fixes the fmt step in the build script for directories.
* Take advantage of coercing anonymous struct literals to struct types.
* Reworks Module to favor Zig source as the primary use case.
Breaks ZIR compilation, which will have to be restored in a future commit.
* Decl uses src_index rather then src, pointing to an AST Decl node
index, or ZIR Module Decl index, rather than a byte offset.
* ZIR instructions have an `analyzed_inst` field instead of Module
having a hash table.
* Module.Fn loses the `fn_type` field since it is redundant with
its `owner_decl` `TypedValue` type.
* Implement Type and Value copying. A ZIR Const instruction's TypedValue
is copied to the Decl arena during analysis, which allows freeing the
ZIR text instructions post-analysis.
* Don't flush the ELF file if there are compilation errors.
* Function return types allow arbitrarily complex expressions.
* AST->ZIR for function calls and return statements.
* Introduce the concept of anonymous Decls
* Primitive Hello, World with inline asm works
* There is still an unsolved problem of how to manage ZIR instructions
memory when generating from AST. Currently it leaks.
One of the main motivating use cases for this language feature is
tracing/profiling tools, which expect null-terminated strings for these
values. Since the data is statically allocated, making them
additionally null-terminated comes at no cost.
This prevents the requirement of compile-time code to convert to
null-termination, which could increase the compilation time of
code with tracing enabled.
See #2029
std.log provides 8 log levels and corresponding logging functions. It
allows the user to override the logging "backend" by defining root.log
and to override the default log level by defining root.log_level.
Logging functions accept a scope parameter which allows the implementer
of the logging "backend" to filter logging by library as well as level.
Using the standardized syslog [1] log levels ensures that std.log will
be flexible enough to work for as many use-cases as possible. If we were
to stick with only 3/4 log levels, std.log would be insufficient for
large and/or complex projects such as a kernel or display server.
[1]: https://tools.ietf.org/html/rfc5424#section-6.2.1
Branch handling `*[N]T` to `E![]T` is already handled in a more complete
branch handling `*[N]T` to `[]T` *and* `*[N]T` to `E![]T` so it seems
safe to remove this one.
- stderr_file_writer was unused
- stderr_stream was a pointer to a stream, rather than a stream
- other functions assumed that getStderrStream has already been called
This will allow the developer to request additional memory pages
from the runtime to be allocated for the Wasm app. Typical usage:
```zig
var wasm_pages = @wasmMemorySize();
@wasmMemoryGrow(1);
@import("std").debug.assert((wasm_pages + 1) == @wasmMemorySize());
```
This will allow the developer to poll the runtime for currently
allocated memory in the number of Wasm pages. Typical usage:
```zig
var wasm_pages = @wasmMemorySize();
@import("std").debug.assert(wasm_pages > 0);
```
Start implementing https://github.com/ziglang/zig/issues/4917 which is to rename instream/outstream to reader/writer. This first change allows code to use Writer/writer instead of OutStream/outStream, but still maintains the old outstream names with "Deprecated" comments.
This commit adds a `--stack [size]` link-time option to zig compiler
allowing the user to override the default stack size set for the
specified executable/library format. This is currently limited to
ELF, COFF and Wasm however (i.e., Mach-O is excluded).
This commit increases the default Wasm stack to 1MB from the default
of 1 Wasm page which equal 64KB. This seems like a reasonable default
size while at the same time not overly large. Also, Rust lang seems
to be favouring this default as well: [rust-lang#50083].
[rust-lang#50083]: https://github.com/rust-lang/rust/pull/50083
The AddressList returned can contain more than one item
e.g. the ipv4 and ipv6 addresses for a given hostname.
Previously if a server had multiple addresses but
was not listening on one of them Zig would give up
immediately.
Now on std.os.ConnectError.ConnectionRefused Zig will
try the next address in the list. Zig still gives up on
all other errors as they are related to the system and
system resources rather than whether the remote server
is listening on a particular address.
- for one-possible-value types, ir_analyze_struct_field_ptr()
no longer hardcodes const/volatile
- when slicing arrays, ir_analyze_instruction_slice()
no longer consults ConstValSpecialStatic
closes#5474
ERROR_DIRECTORY (267) is returned from kernel32.RemoveDirectoryW if the path is not a directory. Note also that os.DirectDirError already includes NotDir
Before: error.Unexpected: GetLastError(267): The directory name is invalid.
After: error: NotDir
generalizes functionality of ArrayList.insertSlice() to overwrite
a range of elements in the list and to grow or shrink the list as needed
to accommodate size difference of the replacing slice and the range
of existing elements.
* improve docs
* add TODO comments for things that don't have open issues
* remove redundant namespacing of struct fields
* guard against ioctl returning EINTR
* remove the general std.os.ioctl function in favor of the specific
ioctl_SIOCGIFINDEX function. This allows us to have a more precise
error set, and more type-safe API.
When using C libraries, C99 designator list initialization is often
times used to initialize data structure.
While `std.mem.zeroes` and manually assigning to each field can
achieve the same result, it is much more verbose then the equivalent
C code:
```zig
usingnamespace @cImport({
@cInclude("sokol_app.h");
});
// Using `std.mem.zeroes` and manual assignment.
var app_desc = std.mem.zeroes(sapp_desc);
app_desc.init_cb = init;
app_desc.frame_cb = frame;
app_desc.cleanup_cb = cleanup;
app_desc.width = 400;
app_desc.height = 300;
app_desc.window_name = "no default init";
// Using `std.mem.defaultInit`.
var app_desc = std.mem.defaultInit(sapp_desc, .{
.init_cb = init,
.frame_cb = frame,
.cleanup_cb = cleanup,
.width = 400,
.height = 300,
.window_name = "default init"
});
```
The `std.mem.defaultInit` aims to solve this problem by zero
initializing all fields of the given struct to their zero, or default
value if any. Each field mentionned in the `init` variable is then
assigned to the corresponding field in the struct.
If a field is a struct, and an initializer for it is present, it is
recursively initialized.
`git add --renormalize .`
For text files, git expects that all files are commited with LF line endings,
it then (optionally) swaps to CRLF on checkout depending on .gitattributes and
git config.
extracted function ir_try_evaluate_bin_op_const
extracted type_is_self_comparable function
renamed ir_try_evaluate_bin_op_const to ir_try_evaluate_bin_op_cmp_const
implemented analysis of ?T == T
added ir_set_cursor_at_end_and_append_basic_block_gen
use build_br_gen and ir_set_cursor_at_end_and_append_block_gen
added ir_append_basic_block_gen
removed include of all_types in ir.cpp
extracted compile-time and runtime evaluation of cmp_optional_non_optional to separate functions
closes#5390closes#1332
Given that the previous design would require the use of a default
allocator to have `ArgIterator.init()` work in WASI, and since in
Zig we're trying to avoid default allocators, I've changed the design
slightly in that now `init()` is a compile error in WASI, and instead
in its message it points to `initWithAllocator(*mem.Allocator)`.
The latter by virtue of requiring an allocator as an argument can
safely be used in WASI as well as on other OSes (where the allocator
argument is simply unused). When using `initWithAllocator` it is then
natural to remember to call `deinit()` after being done with the
iterator. Also, to make use of this, I've also added `argsWithAllocator`
function which is equivalent to `args` minus the requirement of supplying
an allocator and being fallible.
Finally, I've also modified the WASI only test `process.ArgWasiIterator`
to test all OSes.
This commit pulls WASI specific implementation of args extraction
from the runtime from `process.argsAlloc` and `process.argsFree`
into a new iterator struct `process.ArgIteratorWasi`. It also
integrates the struct with platform-independent `process.ArgIterator`.
* Deleted decls are deleted; unused decls are also detected as deleted.
Cycles are not yet detected.
* Re-analysis is smarter and will not cause a re-analysis of dependants
when only a function body is changed.
The binary file abstraction changed its struct named "Decl" to
"TextBlock" and it now represents an allocated slice of memory in
the .text section. It has two new fields: prev and next, making it
a linked list node. This allows a TextBlock to find its neighbors.
The ElfFile struct now has free_list and last_text_block fields.
Doc comments for free_list are reproduced here:
A list of text blocks that have surplus capacity. This list can have false
positives, as functions grow and shrink over time, only sometimes being added
or removed from the freelist.
A text block has surplus capacity when its overcapacity value is greater than
minimum_text_block_size * alloc_num / alloc_den. That is, when it has so
much extra capacity, that we could fit a small new symbol in it, itself with
ideal_capacity or more.
Ideal capacity is defined by size * alloc_num / alloc_den.
Overcapacity is measured by actual_capacity - ideal_capacity. Note that
overcapacity can be negative. A simple way to have negative overcapacity is to
allocate a fresh text block, which will have ideal capacity, and then grow it
by 1 byte. It will then have -1 overcapacity.
The last_text_block keeps track of the end of the .text section.
Allocation, freeing, and resizing decls are all now more sophisticated,
and participate in the virtual address allocation scheme. There is no
longer the possibility for virtual address collisions.
I'm not sure why I disabled them when landing extended Wasm/WASI
support, but they pass the parser tests just fine now, so I'm gonna
go ahead and re-enable them.
* change miscellaneous things to more idiomatic zig style
* change the digest length to 24 bytes instead of 48. This is
still 70 more bits than UUIDs. For an analysis of probability of
collisions, see:
https://en.wikipedia.org/wiki/Universally_unique_identifier#Collisions
* fix the API having the possibility of mismatched allocators
* fix some error paths to behave properly
* modify the guarantees about when file contents are loaded for input files
* pwrite instead of seek + write
* implement isProblematicTimestamp
* fix tests with regards to a working isProblematicTimestamp function.
this requires sleeping until the current timestamp becomes
unproblematic.
* introduce std.fs.File.INode, a cross platform type abstraction
so that cache hash implementation does not need to reach into std.os.
People using the API as intended would never trigger this assertion
anyway, but if someone has a non standard use case, I see no reason
to make the program panic.
If a user doesn't care that the manifest failed to be written, they can
simply ignore it. The program will still work; that particular cache
item will simply not be cached.
It checks whether the cache will respond correctly to inputs that don't
initially depend on filesystem state. In that case, we have to check
for the existence of a manifest file, instead of relying on reading the
list of entries to tell us if the cache is invalid.
Instead of releasing the manifest file when an error occurs, it is
only released when when `CacheHash.release` is called. This maps better
to what a zig user expects when they do `defer cache_hash.release()`.
A file handle is not the same thing as an inode index number.
Eventually the inode will be checked as well, but there needs to be
a way to get the inode in `std` first.
Remove the constants that assume a base unit in favor of explicit
x_per_y constants.
nanosecond calendar timestamps now use i128 for the type. This affects
fs.File.Stat, std.time.nanoTimestamp, and fs.File.updateTimes.
calendar timestamps are now signed, because the value can be less than
the epoch (the user can set their computer time to whatever they wish).
implement std.os.clock_gettime for Windows when clock id is
CLOCK_CALENDAR.
To prevent cache misses, token ids go in their own array, and the
start/end offsets go in a different one.
perf measurement before:
2,667,914 cache-misses:u
2,139,139,935 instructions:u
894,167,331 cycles:u
perf measurement after:
1,757,723 cache-misses:u
2,069,932,298 instructions:u
858,105,570 cycles:u
The DocComment AST node now only points to the first doc comment token.
API users are expected to iterate over the following tokens directly.
After this commit there are no more linked lists in use in the
self-hosted AST API.
Performance impact is negligible. Memory usage slightly reduced.
* Extract Call ast node tag out of SuffixOp; parameters go in memory
after Call.
* Demote AsmInput and AsmOutput from AST nodes to structs inside the
Asm node.
* The following ast nodes get their sub-node lists directly following
them in memory:
- ErrorSetDecl
- Switch
- BuiltinCall
* ast.Node.Asm gets slices for inputs, outputs, clobbers instead of
singly linked lists
Performance changes:
throughput: 72.7 MiB/s => 74.0 MiB/s
maxrss: 72 KB => 69 KB (nice)
block statements are now directly following the Block AST node rather
than a singly linked list. This had negligible impact on performance:
throughput: 72.3 MiB/s => 72.7 MiB/s
however it greatly improves the API since the statements are laid out in
a flat array in memory.
These SuffixOp nodes have their own ast.Node tags now:
* ArrayInitializer
* ArrayInitializerDot
* StructInitializer
* StructInitializerDot
Their sub-expression lists are general-purpose-allocator allocated
and then copied into the arena after completion of parsing.
throughput: 72.9 MiB/s => 74.4 MiB/s
maxrss: 68 KB => 72 KB
The API is also nicer since the sub expression lists are now flat arrays
instead of singly linked lists.
Instead of being its own node, it's a struct inside FnProto.
Instead of FnProto having a SinglyLinkedList of ParamDecl nodes,
ParamDecls are appended directly in memory after the FnProto.
throughput: 72.2 MiB/s => 72.9 MiB/s
maxrss: 70 KB => 68 KB
Importantly, the API is improved as well since the data is arranged
linearly in memory.
This makes fields and decl ast nodes part of the Root and ContainerDecl
AST nodes.
Surprisingly, it's a performance regression from using a singly-linked
list for these nodes:
throughput: 76.5 MiB/s => 69.4 MiB/s
However it has much better memory usage:
maxrss: 392 KB => 77 KB
It's also better API for consumers of the parser, since it is a flat
list in memory.
std.ast uses a singly linked list for lists of things. This is a
breaking change to the self-hosted parser API.
std.ast.Tree has been separated into a private "Parser" type which
represents in-progress parsing, and std.ast.Tree which has only
"output" data. This means cleaner, but breaking, API for parse results.
Specifically, `tokens` and `errors` are no longer SegmentedList but a
slice.
The way to iterate over AST nodes has necessarily changed since lists of
nodes are now singly linked lists rather than SegmentedList.
From these changes, I observe the following on the
self-hosted-parser benchmark from ziglang/gotta-go-fast:
throughput: 45.6 MiB/s => 55.6 MiB/s
maxrss: 359 KB => 342 KB
This commit breaks the build; more updates are necessary to fix API
usage of the self-hosted parser.
However there does not appear to be an x86 encoding for calling an
immediate address. So there's no point of setting this up. We should
just emit an indirect call to the got addr.
This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.
Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.
The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
After this commit there are no more bit rotted files.
The testing program that was in ir.zig has been moved to main.zig
Unsupported command line options have been deleted, or error messages
added.
The compiler repl is available from the build-exe, build-lib,
build-obj commands with the --watch option.
The main zig build script now builds the self-hosted compiler
unconditionally. Linking against LLVM is behind a -Denable-llvm
flag that defaults to off.
* add TypedValue.Managed which represents a Type, a Value, and some
kind of memory management strategy.
* introduce an analysis queue
* flesh out how incremental compilation works with respect to exports
* ir.text.Module is only capable of one error message during parsing
* link.zig no longer has a decl table map and instead has structs that
exist directly on ir.Module.Decl and ir.Module.Export
* implement primitive .text block allocation
* implement linker code for updating Decls and Exports
* implement null Type
Some supporting std lib changes:
* add std.ArrayList.appendSliceAssumeCapacity
* add std.fs.File.copyRange and copyRangeAll
* fix std.HashMap having modification safety on in ReleaseSmall builds
* add std.HashMap.putAssumeCapacityNoClobber
* Add an upper case variant of `allocLowerString`
* Add case-sensitive variants of `eqlIgnoreCase`, `indexOfIgnoreCase`,
and `indexOfIgnoreCasePos`
* Add and update docstrings on functions
* introduce std.ArrayListUnmanaged for when you have the allocator
stored elsewhere
* move std.heap.ArenaAllocator implementation to its own file. extract
the main state into std.heap.ArenaAllocator.State, which can be
stored as an alternative to storing the entire ArenaAllocator, saving
24 bytes per ArenaAllocator on 64 bit targets.
* std.LinkedList.Node pointer field now defaults to being null
initialized.
* Rework self-hosted compiler Package API
* Delete almost all the bitrotted self-hosted compiler code. The only bit
rotted code left is in main.zig and compilation.zig
* Add call instruction to ZIR
* self-hosted compiler ir API and link API are reworked to support
a long-running compiler that incrementally updates declarations
* Introduce the concept of scopes to ZIR semantic analysis
* ZIR text format supports referencing named decls that are declared
later in the file
* Figure out how memory management works for the long-running compiler
and incremental compilation. The main roots are top level
declarations. There is a table of decls. The key is a cryptographic
hash of the fully qualified decl name. Each decl has an arena
allocator where all of the memory related to that decl is stored.
Each code block has its own arena allocator for the lifetime of
the block. Values that want to survive when going out of scope in
a block must get copied into the outer block. Finally, values must
get copied into the Decl arena to be long-lived.
* Delete the unused MemoryCell struct. Instead, comptime pointers are
based on references to Decl structs.
* Figure out how caching works. Each Decl will store a set of other
Decls which must be recompiled when it changes.
This branch is still work-in-progress; this commit breaks the build.
Previously, the path and preopens helpers were prototyped in `std.os.wasi`
module, but since they are higher-level abstraction over wasi, they belong in
`std.fs.wasi` module.
This commit removes `std.os.openatWasi` function, and renames it to
`std.os.wasi.openat`. Additionally, the added `PreopenList.findByPath`
method allows querying the list for a matching preopen by path.
This commit refactors `std.os.wasi.resolve_preopen` into a (higher-level)
`std.os.wasi.getPreopens` funtion which returns a slice with _all_
preopens at any given time. This fn allows the WASI module to
inquire at any given time for all preopens provided by the runtime.
This commit also makes `cwd()` a compile error on WASI.
This commit adds WASI specific impl of `std.fs.cwd()` in which we
emulate the `cwd` behaviour by inquiring the runtime for a "."
preopen if available. This is OK for simple relative ops, but will
not work for any ops which require absolute paths.
It seems that `std.os.openZ` is too POSIX-specific, so I think it
should not be a point of entry for WASI `open` call. I figure
WASI should be treated as a separate "os" that's _not_ POSIX
especially given the incoming changes in the ephemeral snapshot.
According to documentation ETIMEDOUT (110) is a valid error code for the read function. I just had my long-running (been running for about 7 weeks) network program crash because it did not handle the ETIMEDOUT error code from "read".
Before it was possible for .intended_io_mode = .blocking,
.capable_io_mode = .evented, and then the implementation would put a
request on the fs thread, which is the wrong behavior. Now it always
calls the appropriate WriteFile/ReadFile function, passing the intended
io mode directly as a parameter.
This makes the behavior tests pass on Windows with --test-evented-io.
Now there are 3 types:
* std.math.big.int.Const
- the memory is immutable, only stores limbs and is_positive
- all methods operating on constant data go here
* std.math.big.int.Mutable
- the memory is mutable, stores capacity in addition to limbs and
is_positive
- methods here have some Mutable parameters and some Const
parameters. These methods expect callers to pre-calculate the
amount of resources required, and asserts that the resources are
available.
* std.math.big.int.Managed
- the memory is mutable and additionally stores an allocator.
- methods here perform the resource calculations for the programmer.
- this is the high level abstraction from before
Each of these 3 types can be converted to the other ones.
You can see the use case for this in the self-hosted compiler, where we
only store limbs, and construct the big ints as needed.
This gets rid of the hack where the allocator was optional and the
notion of "fixed" versions of the struct. Such things are now modeled
with the `big.int.Const` type.
`wasi_snapshot_preview1` introduced a couple of ABI changes. This
commit fast-forwards the types and consts defined in `std.os.bits.wasi`
to match those changes.
This commit updates the WASI imports to use `wasi_snapshot_preview1`
instead of the old `wasi_unstable`. There are some minor ABI
differences between the two, however, the main motivator for using
the latest "stable" snapshot (aka preview1) is that, at least in
Wasmtime, there has been a lot of improvement work put into preview1
and unfortunately I might add, the improvements were not (in full)
backported to `wasi_unstable` snapshot.
Also, this commit removes the optional bound on the pointer to
`environ_get` syscall.
Pre-requisite for having a test case for #5062
In complex C statements which are outside of macros,
it is valid C to perform e.g. a bitor between an
integer and a boolean `5 | (8 == 9)`
Currently this results in a zig error after translating
as `c_int | bool` is invalid Zig.
Detects if a sub-expression of a numeric operator is
boolean and if so converts it to int
See https://github.com/ifreund/river/issues/17 for an issue that occurs
because the field names are mangled globally. When using the generated
bindings, you have no choice but to use the unstable names or redeclare
the entire struct. This commit changes the behaviour to use a local
counter per record declaration, so the names are predictable each time.
* add tests for negative inputs to cosh32 and cosh64. fix bug in cosh64 for negative inputs.
* fix problem with negative input with f64 sinh and add tests
Jonathan S writes:
On common systems with a 022 umask, this will still result in a
file created with 755 permissions, but it works appropriately if the
system is configured more leniently. (As another data point, C's fopen
seems to open files with the 666 mode.)
Consider a (legal according to the `@bitCast` rules) conversion from u16
to [2]u8: since the former is a scalar and the latter is a pointer
(arrays are represented at pointers in the codegen phase) we have to
allocate a temporary slot on the stack and then bitcast the resulting
pointer to the desired destination type.
Beware that this means the lifetime of the resulting value is the same
of the function it's contained in and for all intents and purposes
should be regarded as a local (eg. it should not escape).
Closes#4395Closes#5121
Fallback to sysctl `kern.osversion` when `kern.osproductversion` is not
available (prior to 10.13.4) .
The mapping from `sw_vers -buildVersion` to `-productVersion` is
formulaic from 10.8 to 10.15 and older is handled with switch.
closes#5119
Ziggurat rng was using deprecated `random.scalar(u64)` which was causing compile errors on calls to public facing stdlib APIs (randExp) on 0.6+, this fixed those errors.
The codegen would sometimes change the LLVM type for some constants to
an unnamed structure in order to accomodate extra padding. This is fine
as long as the alignment of each field is still respected and it was not
the case for structure types, leading to ill-formed constants being
generated.
Optional types suffer from this to a lower extent as their layout is
quite lucky, the only missing piece was the tail padding.
Closes#4530Closes#4594Closes#4295
Closes my will to live
fff3c1fff4 broke the docs generation for
own user code. This adds a workaround, because all the code in docs/main.js
is relying on the fact that the root package of the main package is the
root package itself.
Without this, building from source caused:
CommandLine Error: Option 'mc-relax-all' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
This is due to LLVM static libs compiled in multiple times. But without
the LLVM static libs on the linker line, it caused undefined symbol
linker errors.
So our hands are tied. Homebrew users will have to specify
`-DZIG_PREFER_CLANG_CPP_DYLIB`.
The tests were put into a deadlock, and it seems that darwin doesn't
support `O_SYNC`, though it supports `O_NONBLOCK`. It shouldn't block
even with that, but I'm not sure why else it would fail.
The share_access bitfield was being ORed with what was supposed to be
parts of the default value, meaning that the share_access would be
more permissive than expected.
* don't unconditionally pass -lz3 for mingw builds. If mingw builds
require this then the llvm-config executable should put it as part of
--system-libs. If there is a bug and it does not do that, and we need
a workaround, then the workaround should be an explicit cmake option.
* don't link libstage2.a against -lntdll. This causes zig to set
`builtin.link_mode == .Dynamic` and include the TLS definitions,
which then collide with the mingw-w64 symbols. This should probably
be addressed separately, but for now this solves the problem and
there is no reason to link a static library against a DLL.
* Findllvm.cmake no longer treats the libraries as "optional" and will
emit a cmake error if one is not found. Additionally, the
not-required LLVM library LLVMTableGen is omitted.
Whether the C ABI is mingw-w64 or msvc, detection of native libc paths
should be the same. In the future we may want to allow passing a C ABI
parameter to detectNativeCPaths() but for now we have the same behavior
regardless.
The v6m ISA has no way to express a CAS loop natively without turning
off the interrupts or using the kernel cmpxchg harness.
On such a platform the user has to provide a few __sync_* builtins to
satisfy the linker.
Before, type_has_one_possible_value would return false for the value
`.{1}`. But actually, that type is a tuple with a single comptime field.
Such a type, in fact, has one possible value.
This plus the corresponding adjustment to get_the_one_possible_value
solves #3878.
Rather than stuffing configuration information into the Zig binary, the
build script reads it from config.h. This solves a problem for package
maintainers and improves the use case of deterministic builds.
closes#3758
Before, this would cause a link failure when mixing Zig and C code for
RISC-V targets.
Now, the ABIs match and Zig and C code can be mixed successfully.
I will file a follow-up issue for the ability to deal more explicitly
with ABIs.
closes#4863
`flock` locks based on the file handle, instead of the process id.
This brings the file locking on unix based systems closer to file
locking on Windows.
In an MSVC context, `-MT` means
"Use static run-time"
and it is a flag with no parameter.
On POSIX it means
"Specify name of main file output in depfile"
and it is "joined or separate".
The former was interfering with the latter. Now, the MT flag is required
to be specified with a `/` to disambiguate: `/MT`.
Now, this function first attempts a case-sensitive lookup.
If no match is found, and `key` is ASCII, then it attempts a
second case-insensitive lookup.
It is not planned to support full Unicode case-insensitivity
on Windows, and in fact relying on non-ASCII case-insensitive
environment variables is fundamentally problematic.
Make fallthrough an error when compiler supports it. This requires a new
macro that is defined with such compilers to be used as a statement, at
all fallthrough sites:
switch (...) {
case 0:
...
ZIG_FALLTHROUGH;
case 1:
...
break;
default:
...
break;
}
If we ever move to C++17 as minimal requirement, then the macro can be
replaced with `[[fallthrough]];` at statement sites.
This new name (and the fact that it is a function returning a type) will
make it more clear which use cases are better suited for ArrayList and
which are better suited for ArrayListSentineled.
Also for consistency with ArrayList,
* `append` => `appendSlice`
* `appendByte` => `append`
Thanks daurnimator for pointing out the confusion of std.Buffer.
- Fix undeclared variable in 'writeItem'
- Clarify docs of `read` regarding bytes vs. items
- Normalize 'writeable' to 'writable' (the more common parlance)
This caused link errors in c++ code because it was not correct to pass
these flags to child codegens. And that was the only reason to detect
these flags. Otherwise we can safely rely on non-explicitly-detected
flag forwarding.
closes#4766
This commit also fixes the implementation of some utility functions for
adjusting properties of pointer types. Previously these functions would
incorrectly drop vector, sentinel, and inference metadata.
Constant-folding the pointers to the expected sentinel value have some
big problems: it hides the real content of the array, makes the pointer
to the sentinel point to a completely different memory region and treats
it like a const value even when the underlying array is mutable.
Fixes#4840
before:
```
std\net.zig:403:23: error: type '@TypeOf(std.net.getAddressList).ReturnType.ErrorSet!*std.net.AddressList' does not support field access
const addrs = list.addrs.toSliceConst();
^
```
* Always allocate an info block per-thread so that libc can store
important stuff there.
* Respect ABI-mandated alignment in more places.
* Nicer code, use slices/pointers instead of raw addresses whenever
possible.
This is only the first step; it makes zig cc recognize -z and append it to the linker args, but the linker arg parsing doesn't support -z yet so it will just give the warning 'unsupported linker arg: -z'
This reverts commit ee6fda2297, reversing
changes made to f313ab18ae.
This caused a test failure:
```
behavior.misc.test "behavior-arm-linux-none-Debug-bare-multi thread local variable"...test failure
/home/vsts/work/1/s/lib/std/testing.zig:191:14: 0x4608f in std.testing.expect (test)
if (!ok) @panic("test failure");
^
/home/vsts/work/1/s/test/stage1/behavior/misc.zig:616:11: 0x53e93 in behavior.misc.test "behavior-arm-linux-none-Debug-bare-multi thread local variable" (test)
expect(S.t == 1235);
^
```
* `-isystem` instead of `-I` for system include directories
fixes a problem with native system directories interfering with zig's
bundled libc.
* separate Stage2Target.is_native into Stage2Target.is_native_os and
Stage2Target.is_native_cpu.
Match package-name case from CMakeLists.txt .
New warning sample:
The package name passed to `find_package_handle_standard_args` (LLVM) does
not match the name of the calling package (llvm). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
* Always allocate an info block per-thread so that libc can store
important stuff there.
* Respect ABI-mandated alignment in more places.
* Nicer code, use slices/pointers instead of raw addresses whenever
possible.
On NetBSD this is needed to avoid crashes in pthread_join as the default
value for the guard page size is not ignored even though a custom stack
address is specified.
* Make the tokenizer spit out an Invalid token on the first invalid
character found in the number literal.
* More parsing and tokenizer tests for number literals
* fix invalid switch statement in ir.zig
* The generated options data file is sorted now in a way that
makes sure longer prefixes are first. This prevents collisions
with some parameters.
* Add support for `-fPIC`, `-fno-PIC`, `-nostdlib`, `-shared`,
`-rdynamic`, `-Wl,-soname`, `-Wl,-rpath`
* Better support for `-o`.
* Disable generating h files
* Shared library support.
* Better positional argument support.
because it uncovered a result location bug, and I need to get this
branch merged before going into a result location rabbit hole.
also fix the result type of slicing when the indexes are runtime known
and the result should be sentinel terminated.
* Don't add an extra slot for the sentinel. Most of the code keeps using
the constant value from the type descriptor, let's harmonize all the
code dealing with sentinels.
* Properly write out sentinel values when reinterpreting pointers at
comptime.
* Allow the reading of the 0th element in a `[0:S]T` type.
Remove `std.fs.deleteTree`. Callers instead should use
`std.fs.cwd().deleteTree`.
Add `std.fs.deleteTreeAbsolute` for when the caller has an absolute
path.
* remove deprecated `std.fs.Dir` APIs
* `std.fs.Dir.openDir` now takes a options struct with bool fields for
`access_sub_paths` and `iterate`. It's now much more clear how
opening directories works.
* fixed the std lib and various zig code calling the wrong openDir
function.
* the runtime safety check for dir flags is removed in favor of the
cheaper option of putting a comment on the same line as handling
EBADF / ACCESS_DENIED, since that will show up in stack traces.
closes#4682
The self-hosted compiler is still bit rotted and still not compiling
successfully yet. I have a more serious rework of the code in a
different branch.
This behaves the same as stdout and stderr behavior which also
default to .inherit. Also adds a field to RunStep to change the behavior.
Since this is a breaking change, previous behavior can be restored by doing:
`RunStep.stdin_behavior = .Ignore`.
* Underscores `_` may be placed between two digits in a int/float literal
* Consecutive underscores are not allowed
* Fixed parsing bug in exponents of hexadecimal float literals.
Exponents should always be base 10, but hex characters would be parsed
inside the exponent and everything after them would be ignored. eg:
`0x1.0p1ab1` would be parsed as `0x1.0p1`.
When building an object file from only one source file, instead of
having a two-stage cache system, we special case it and use the cache
directory that the .o file is output to as the final cache directory for
all the build artifacts.
When there are more than 1 source file, the linker has to merge objects
into one, and so the two stage approach makes sens. But in the case of
only one source file, this prevents needlessly copying the object file.
This commit fixes an issue with the previous one, where zig with cache
enabled would print a directory that actually did not have any build
artifacts in it.
Previously the zig build system incorrectly assumed that the only build
artifact was a binary. Now, when you enable the cache, only the output
dir is printed to stdout, and the zig build system iterates over the
files in that directory, copying them to the output directory.
To support this change:
* Add `std.os.renameat`, `std.os.renameatZ`, and `std.os.renameatW`.
* Fix `std.os.linux.renameat` not compiling due to typos.
* Deprecate `std.fs.updateFile` and `std.fs.updateFileMode`.
* Add `std.fs.Dir.updateFile`, which supports using open directory
handles for both the source and destination paths, as well as an
options parameter which allows overriding the mode.
* Update `std.fs.AtomicFile` to support operating based on an open
directory handle. Instead of `std.fs.AtomicFile.init`, use
`std.fs.Dir.atomicFile`.
* `std.fs.AtomicFile` deinit() better handles the situation when the
rename fails but the temporary file still exists, by still
attempting to remove the temporary file.
* `std.fs.Dir.openFileWindows` is moved to `std.os.windows.OpenFileW`.
* `std.os.RenameError` gains the error codes `NoDevice`,
`SharingViolation`, and `PipeBusy` which have been observed from
Windows.
Closes#4733
* `std.Buffer.print` is removed; use `buffer.outStream().print`
* `std.fmt.count` returns a `u64`
* `std.Fifo.print` is removed; use `fifo.outStream().print`
* `std.fmt.bufPrint` error is renamed from `BufferTooSmall`
to `NoSpaceLeft` to match `std.os.write`.
* `std.io.FixedBufferStream.getWritten` returns mutable buffer
if the buffer is mutable.
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.
Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.
See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.
See #130 for the efforts to make these abstractions non-generic.
This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
* Fix codegen for optional types that decay to a pointer, the type
behaves as a boolean
* Fix comptime evaluation of zero-sized arrays, always initialize the
internal array elements
Closes#4673
Translate C: Put an alignCast in c style pointer casts to allow opaque types to cast properly in C macros
Translate C: add test case for aligning opaque types in pointer casts
Translate C: Fix @typeId -> @typeInfo
Add test case to run_translated_c for casting from pointer to opaque type
Add std.Target.Cpu.Model.generic which is even more empty than baseline.
CPU model and feature detection uses this rather than baseline.
Rename cpu_detected to cpu_detection_unimplemented and flip the logic.
It can be relied on by stage2.zig to decide whether the LLVM workaround
is needed without also checking the CrossTarget.
Move the CPU detection to after the OS detection, and use the detected
OS for the CPU detection. This is relevant because operating systems
sometimes emulate certain CPU features, so knowing the OS and version is
relevant for determining CPU features.
Prepare for #4592 by passing the CPU arch to the detection code, instead
of having it rely on Target.current.
The CPU model & feature detection logic is modified. Before:
* Detect actual features
* Use as hint when detecting CPU model
* Populate dependencies of CPU model features
* Merge that into the actual features set
After:
* Detect actual features
* Use as hint when detecting CPU model
* Add known CPU model features to actual features
* Detect actual features again, overriding known CPU model features
* Populate dependencies
Now it will emit a compile error rather than crashing when the child
type has not been resolved properly.
Introduces `get_optional_type2` which should be used generally inside
ir.cpp.
Fix some std lib compile errors noticed by the provided test case.
Thanks @LemonBoy for the test case. Closes#4377.
Fixes#4374.
Apparently NtCurrentTeb is only exported for i386 and some other
platforms but not for x86_64 nor AArch64. Let's go with the flow and
provide our own NtCurrentTeb like the Windows headers do.
Thank you Microsoft.
And fix most of the fallout. This also makes optional pointers not
require resolving zero bits, because the comptime value struct layout no
longer depends on whether the type has zero bits.
Thanks to @LemonBoy for the behavior test case
Closes#4357Closes#4359
* improve `std.fs.AtomicFile` to use sendfile()
- also fix AtomicFile cleanup not destroying tmp files under some
error conditions
* improve `std.fs.updateFile` to take advantage of the new `makePath`
which no longer needs an Allocator.
* rename std.fs.makeDir to std.fs.makeDirAbsolute
* rename std.fs.Dir.makeDirC to std.fs.Dir.makeDirZ
* add std.fs.Dir.makeDirW and provide Windows implementation of
std.os.mkdirat. std.os.windows.CreateDirectory is now implemented
by calling ntdll, supports an optional root directory handle,
and returns an open directory handle. Its error set has a few more
errors in it.
* rename std.fs.Dir.changeTo to std.fs.Dir.setAsCwd
* fix std.fs.File.writevAll and related functions when len 0 iovecs
supplied.
* introduce `std.fs.File.writeFileAll`, exposing a convenient
cross-platform API on top of sendfile().
* `NoDevice` added to std.os.MakeDirError error set.
* std.os.fchdir gets a smaller error set.
* std.os.windows.CloseHandle is implemented with ntdll call rather than
kernel32.
The previous behaviour of using path.resolve has unexpected behaviour around symlinks.
This more simple implementation is more correct and doesn't require an allocator
* rework os.sendfile and add macosx support, and a fallback
implementation for any OS.
* fix sendto compile error
* std.os write functions support partial writes. closes#3443.
* std.os pread / pwrite functions can now return `error.Unseekable`.
* std.fs.File read/write functions now have readAll/writeAll variants
which loop to complete operations even when partial reads/writes
happen.
* Audit std.os read/write functions with respect to Linux returning
EINVAL for lengths greater than 0x7fff0000.
* std.os read/write shim functions do not unnecessarily loop. Since
partial reads/writes are part of the API, the caller will be forced
to loop anyway, and so that would just be code bloat.
* Improve doc comments
* Add a non-trivial test for std.os.sendfile
* Fix std.os.pread on 32 bit Linux
* Add missing SYS_sendfile bit on aarch64
This changset adds a `sendfile(2)` syscall bindings to the linux bits
component. Where available, the `sendfile64(2)` syscall will be
transparently called.
A wrapping function has also been added to the std.os to transform
errno returns to Zig errors.
Change-Id: I86769fc4382c0771e3656e7b21137bafd99a4411
* Slices from C pointers are no longer allowzero but instead insert a runtime assertion.
* Added a test, fixed code for cases with non-allowzero C pointers
* Create new type when flipping allow_zero, sometimes we get a cached value back from adjust_ptr_len.
* Added comments, changed panic message
* Added runtime safety test.
* `std.Target.getStandardDynamicLinkerPath` =>
`std.Target.standardDynamicLinkerPath`
* it now takes a pointer to fixed size array rather than an allocator
* `std.zig.system.NativeTargetInfo.detect` now supports reading
PT_INTERP from /usr/bin/env
Zig now supports a more fine-grained sense of what is native and what is
not. Some examples:
This is now allowed:
-target native
Different OS but native CPU, default Windows C ABI:
-target native-windows
This could be useful for example when running in Wine.
Different CPU but native OS, native C ABI.
-target x86_64-native -mcpu=skylake
Different C ABI but otherwise native target:
-target native-native-musl
-target native-native-gnu
Lots of breaking changes to related std lib APIs.
Calls to getOs() will need to be changed to getOsTag().
Calls to getArch() will need to be changed to getCpuArch().
Usage of Target.Cross and Target.Native need to be updated to use
CrossTarget API.
`std.build.Builder.standardTargetOptions` is changed to accept its
parameters as a struct with default values. It now has the ability to
specify a whitelist of targets allowed, as well as the default target.
Rather than two different ways of collecting the target, it's now always
a string that is validated, and prints helpful diagnostics for invalid
targets. This feature should now be actually useful, and contributions
welcome to further improve the user experience.
`std.build.LibExeObjStep.setTheTarget` is removed.
`std.build.LibExeObjStep.setTarget` is updated to take a CrossTarget
parameter.
`std.build.LibExeObjStep.setTargetGLibC` is removed. glibc versions are
handled in the CrossTarget API and can be specified with the `-target`
triple.
`std.builtin.Version` gains a `format` method.
* re-introduce `std.build.Target` which is distinct from `std.Target`.
`std.build.Target` wraps `std.Target` so that it can be annotated as
"the native target" or an explicitly specified target.
* `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
struct which has the tag as well as version range information.
* `std.elf` gains some more ELF header constants.
* `std.Target.parse` gains the ability to parse operating system
version ranges as well as glibc version.
* Added `std.Target.isGnuLibC()`.
* self-hosted dynamic linker detection and glibc version detection.
This also adds the improved logic using `/usr/bin/env` rather than
invoking the system C compiler to find the dynamic linker when zig
is statically linked. Related: #2084
Note: this `/usr/bin/env` code is work-in-progress.
* `-target-glibc` CLI option is removed in favor of the new `-target`
syntax. Example: `-target x86_64-linux-gnu.2.27`
closes#1907
* Correctly fold ptrToInt on optional types
* Generate null as ConstPtrSpecialNull in intToPtr
* Correctly stop ptrToInt on ?*T where T is zero-sized
Closes#4535
Calling Timer.lap queried the system time twice; once to compute the lap
time and once to reset the timer. This can lead to time discrepancies
between actual and computed durations when summing the result of
Timer.lap in a loop. This commit fixes that.
also fix Timer.read to not require a pointer
std.os.getenv and std.os.getenvZ have nice compile errors when not linking
libc and using Windows.
std.os.getenvW is provided as a Windows-only API that does not require
an allocator. It uses the Process Environment Block.
std.process.getEnvVarOwned is improved to be a simple wrapper on top of
std.os.getenvW.
std.process.getEnvMap is improved to use the Process Environment Block
rather than calling GetEnvironmentVariableW.
std.zig.system.NativePaths uses process.getEnvVarOwned instead of
std.os.getenvZ, which works on Windows as well as POSIX.
This function expands argv[0] into the absolute path resolved with PATH
environment variable before making the execve syscall. However, in case
the execve fails, e.g. with ENOENT, it did not restore argv to how it
was before it was passed in. This resulted in the caller performing an
invalid free.
This commit also adds verbose debug info when native system C compiler
detection fails. See #4521.
When the build.zig logic to build libzigstage2 was converted to a cmake
command, it neglected to use baseline CPU features rather than compiling
with native features.
This adds a hard coded flag `-mcpu=baseline` which can be used to detect
the native target, but mark it as non-native so that it does not get the
CPU features specific to the host used to compile libzigstage2.
Full `-mcpu` support is happening in the upcoming pull request #4509,
and so this "quick fix" will be cleaned up in that branch, before it is
merged to master.
closes#4506
This was deceptive. It was always meant to be sort of a "GNU readline"
sort of thing where it provides a Command Line Interface to input text.
However that functionality did not exist and it was basically a red
herring for people trying to read line-delimited input from a stream.
In this commit the API is deleted, so that people can find the proper
API more easily.
A CLI text input abstraction would be useful but may not even need to be
in the standard library. As you can see in this commit, the guess_number
CLI game gets by just fine by using `std.fs.File.read`.
I think this is working correctly. Without also removing sse2 from the
feature set, sse gets added back into the set because sse2 is part of
the x86_64 baseline (which is the cpu provided) and sse2 depends on sse.
in favor of CPU features. Also rearrange the `std.Target`
data structure.
* note: `@import("builtin")` was already deprecated in favor of
`@import("std").builtin`.
* `std.builtin.arch` is now deprecated in favor of
`std.builtin.cpu.arch`.
* `std.Target.CpuFeatures.Cpu` is now `std.Target.Cpu.Model`.
* `std.Target.CpuFeatures` is now `std.Target.Cpu`.
* `std.Target` no longer has an `arch` field. Instead it has a
`cpu` field, which has `arch`, `model`, and `features`.
* `std.Target` no longer has a `cpu_features` field.
* `std.Target.Arch` is moved to `std.Target.Cpu.Arch` and
it is an enum instead of a tagged union.
* `std.Target.parseOs` is moved to `std.Target.Os.parse`.
* `std.Target.parseAbi` is moved to `std.Target.Abi.parse`.
* `std.Target.parseArchSub` is only for arch now and moved
to `std.Target.Cpu.Arch.parse`.
* `std.Target.parse` is improved to accept CPU name and features.
* `std.Target.Arch.getBaselineCpuFeatures` is moved to
`std.Target.Cpu.baseline`.
* `std.Target.allCpus` is renamed to `std.Target.allCpuModels`.
* `std.Target.defaultAbi` is moved to `std.Target.Abi.default`.
* Significant cleanup of aarch64 and arm CPU features, resulting in
the needed bit count for cpu feature set going from 174 to 138.
* Add `std.Target.Cpu.Feature.Set.addFeatureSet` for merging
feature sets together.
`-target-feature` and `-target-cpu` are removed in favor of
`-mcpu`, to conform to established conventions, and it gains
additional power to support cpu features. The syntax is:
-mcpu=name+on1+on2-off1-off2
closes#4261
* remove "cpu features" that are actually just processors
* rename `v8` to `v8a`. this matches the corresponding
change to target/aarch64.zig
* rename types in preparation for removing sub-architecture
from `std.Target`.
I have other files changed in my dirty working tree, but about to make
some changes to arm.zig that I don't want batched with this commit.
The current target's ABI cannot be relied on for this.
For example, we may build the zig compiler for target
riscv64-linux-musl and provide a tarball for users to
download. A user could then run that zig compiler on
riscv64-linux-gnu. This use case is well-defined and
supported by Zig. But that means that we must detect
the system ABI here rather than
relying on `std.Target.current`.
Some C compilers, such as Clang, are known to rely on
argv[0] to find the path to their own executable,
without even bothering to resolve PATH. This results
in the message:
error: unable to execute command: Executable "" doesn't exist!
So we tell ChildProcess to expand argv[0] to the absolute path
to give them a helping hand.
when both `--cache on` and `--output-dir` parameters
are provided. This prevents re-linking `zig` with every
`make` even when `libzigstage2.a` was unchanged.
Rather than `zig0 build ...` the build now does
`zig0 build-lib ...`, avoiding the requirement of linking the build
script, and thus avoiding the requirement of finding native libc,
for systems where libc is the system ABI.
* libc_installation.cpp is deleted.
src-self-hosted/libc_installation.zig is now used for both stage1 and
stage2 compilers.
* (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
now encourages use with an open directory handle.
* Add `std.os.faccessat` and related functions.
* Deprecate the "C" suffix naming convention for null-terminated
parameters. "C" should be used when it is related to libc. However
null-terminated parameters often have to do with the native system
ABI rather than libc. "Z" suffix is the new convention. For example,
`std.os.openC` is deprecated in favor of `std.os.openZ`.
* Add `std.mem.dupeZ` for using an allocator to copy memory and add a
null terminator.
* Remove dead struct field `std.ChildProcess.llnode`.
* Introduce `std.event.Batch`. This API allows expressing concurrency
without forcing code to be async. It requires no Allocator and does
not introduce any failure conditions. However it is not thread-safe.
* There is now an ongoing experiment to transition away from
`std.event.Group` in favor of `std.event.Batch`.
* `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
slightly more efficient on most systems, and works around a
limitation of `getenv` lack of integration with libc.
* (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
`ReadOnlyFileSystem`. Previously these error codes were all reported
as `PermissionDenied`.
* Add `std.Target.isDragonFlyBSD`.
* stage2: access to the windows_sdk functions is done with a manually
maintained .zig binding file instead of `@cImport`.
* Update src-self-hosted/libc_installation.zig with all the
improvements that stage1 has seen to src/libc_installation.cpp until
now. In addition, it now takes advantage of Batch so that evented I/O
mode takes advantage of concurrency, but it still works in blocking
I/O mode, which is how it is used in stage1.
See discussion here for context:
c6df5deb34 (comments)
Michael - I appreciate what you did here, making the configure script
work better for people in practice. When it was checking the build type
against a whitelist, I think it was worth it. However, now that we are
supporting systems which use non-standard cmake build modes, I don't
think this case-mismatch detection thing is worth it. It's starting to
get to the point where it's a lot of complication for very little
benefit. Besides, cmake is not case sensitive. If we support
non-standard build modes, then we would need to support a hypothetical
build mode of `release` (lower case).
So let's just remove this and rely on people to use the build system
correctly (like they will have to do when building any cmake project
from source).
Various maintainers pass custom build types and we don't need to check
those. We are interested only in checking and diagnosing common errors
for Zig project supported types.
Check is now limited to look for case-mismatch only on the well-known
values { Debug, Release, RelWithDebInfo, MinSizeRel }.
- split util_base.hpp from util.hpp
- new namespaces: `mem` and `heap`
- new `mem::Allocator` interface
- new `heap::CAllocator` impl with global `heap::c_allocator`
- new `heap::ArenaAllocator` impl
- new `mem::TypeInfo` extracts names without RTTI
- name extraction is enabled w/ ZIG_ENABLE_MEM_PROFILE=1
- new `mem::List` takes explicit `Allocator&` parameter
- new `mem::HashMap` takes explicit `Allocator&` parameter
- add Codegen.pass1_arena and use for all `ZigValue` allocs
- deinit Codegen.pass1_arena early in `zig_llvm_emit_output()`
See #3180 for a more comprehensive plan to catch this problem. More
sophisticated control flow analysis is needed to provide compile errors
for returning local variable addresses from a function.
and fix larger-than-one-byte sentinels when being freed
Thank you to João Pedro for identifying both problems and
providing example code to solve them.
closes#4413
1. behavior tests with --test-evented-io
2. std lib tests with --test-evented-io
3. fuzz test evented I/O a bit, make it robust
4. make sure it works on all platforms (kqueue, Windows IOCP,
epoll/other)
5. restart efforts on self-hosted
no-frame-pointer-elim and no-frame-pointer-elim-non-leaf have been
deprecated for a while in favour of the newer (and clearer)
frame-pointer attribute.
Starting with LLVM10 the old attributes are silently ignored, leading to
no stack traces in debug mode.
* `zig test` gainst `--test-evented-io` parameter and gains the ability
to seamlessly run async tests.
* `std.ChildProcess` opens its child process pipe with O_NONBLOCK when
using evented I/O
* `std.io.getStdErr()` gives a File that is blocking even in evented
I/O mode.
* Delete `std.event.fs`. The functionality is now merged into `std.fs`
and async file system access (using a dedicated thread) is
automatically handled.
* `std.fs.File` can be configured to specify whether its handle is
expected to block, and whether that is OK to block even when in
async I/O mode. This makes async I/O work correctly for e.g. the
file system as well as network.
* `std.fs.File` has some deprecated functions removed.
* Missing readv,writev,pread,pwrite,preadv,pwritev functions are added
to `std.os` and `std.fs.File`. They are all integrated with async
I/O.
* `std.fs.Watch` is still bit rotted and needs to be audited in light
of the new async/await syntax.
* `std.io.OutStream` integrates with async I/O
* linked list nodes in the std lib have default `null` values for
`prev` and `next`.
* Windows async I/O integration is enabled for reading/writing file
handles.
* Added `std.os.mode_t`. Integer sizes need to be audited.
* Fixed#4403 which was causing compiler to crash.
This is working towards:
./zig test ../test/stage1/behavior.zig --test-evented-io
Which does not successfully build yet. I'd like to enable behavioral
tests and std lib tests with --test-evented-io in the test matrix in the
future, to prevent regressions.
Clean up the code a bit and introduce a few checks meant to avoid
overshooting the end of the frame chain.
The code is now stable enough not to cause panics during the call frame
walking.
* Prevent the next_param_index to become greater than the param_count
one as it's expected by every other function.
* Fix a typo in a error message.
Closes#4381
- zig CMakeLists.txt CMAKE_BUILD_TYPE string comparisons are case-sensitive
- cmake itself is unclear about how tolerant it is for case-mismatches
- add CMAKE_BUILD_TYPE guard in CMakeLists.txt to force exact matches
I do not see many cases of constant pointers to arrays in the stdlib.
In fact, this makes the code run a little faster, probably because Zig
automatically converts to pointers where it makes sense.
This is a translation of the [official reference implementation][1] with
few other changes. The bad news is that the reference implementation is
designed for simplicity and not speed, so there's a lot of room for
performance improvement. The good news is that, according to the crypto
benchmark, the implementation is still fast relative to the other
hashing algorithms:
```
md5: 430 MiB/s
sha1: 386 MiB/s
sha256: 191 MiB/s
sha512: 275 MiB/s
sha3-256: 233 MiB/s
sha3-512: 137 MiB/s
blake2s: 464 MiB/s
blake2b: 526 MiB/s
blake3: 576 MiB/s
poly1305: 1479 MiB/s
hmac-md5: 653 MiB/s
hmac-sha1: 553 MiB/s
hmac-sha256: 222 MiB/s
x25519: 8685 exchanges/s
```
[1]: https://github.com/BLAKE3-team/BLAKE3
- Strange memory corruption issues occur when allocated memory is passed to the builder and it is `defer`'d and freed
- Instead, `dupe` the string as is done in other handlers, this fixes the issue
this was causing unrelated behavior tests to fail.
if this commit is reverted, the docs are good, but `@newStackCall` is
already deprecated in favor of `@call`, supplying the `stack` property.
* Let's consolidate the special-cased DWARF interpreter for OSX with the
general purpose one
* Drop the assumption that all the debug data is contained in a single
contiguous slice of memory. This is a good news for freestanding
targets and paves the way for supporting compressed debug sections.
Let's always subtract 1 from the return address so that we're sure to be
inside the callee.
Fixes some edge case where the stack trace skipped the first entry.
in stack tracing code, the idea was to detect the tty settings at the
top of the stack and pass the information down. somewhere along the way
this got changed so that setTtyColor was assuming the global stderr_file
was related to the output stream the stack trace was being printed to.
now, tty_color is changed to tty_config, and it is an enum rather than a
bool, telling how tty colors are expected to be handled. windows is
still incorrectly looking at stderr_file.
This makes it so that less memory is used for IR instructions, as well
as catching bugs when one expected one kind of instruction and received
the other.
These were never working with native CPU features. In this branch,
we fix native CPU features not being enabled on Windows, and regress
f128 language features. In the llvm10 branch, all this is fixed,
and the tests are re-enabled.
Previously it was a tagged union which was one of:
* baseline
* a specific CPU
* a set of features
Now, it's possible to have a CPU but also modify the CPU's feature set
on top of that. This is closer to what LLVM does.
This is more correct because Zig's notion of CPUs (and LLVM's) is not
exact CPU models. For example "skylake" is not one very specific model;
there are several different pieces of hardware that match "skylake" that
have different feature sets enabled.
comment from this commit reproduced here:
I have observed the CPU name reported by LLVM being incorrect. On
the SourceHut build services, LLVM 9.0 reports the CPU as "athlon-xp",
which is a 32-bit CPU, even though the system is 64-bit and the reported
CPU features include, among other things, +64bit.
So the strategy taken here is that we observe both reported CPU, and the
reported CPU features. The features are trusted more; but if the features
match exactly the features of the reported CPU, then we trust the reported CPU.
Previously, buffers were used with toOwnedSlice() to create c strings
for LLVM cpu/feature strings. However, toOwnedSlice() shrinks the
string memory to the buffer's length, which cuts off the null terminator.
Now toSliceConst() is used instead, and the buffer is not deinited
so that the string memory is not freed.
use @fieldParentPtr to access your context fields, which lie if you
struct that contains a rb.Tree member (without a pointer).
Also simplifies the init() function so rb.Tree can be initialized in a single line,
without having to use "undefined".
this reverts one part of 4c3bfeca. it solves some behavior regressions
but introduces new ones. This change was incorrect to make however,
and this commit takes the code in a better direction.
Previously, there was hacky code to deal with result locations and how
they work with regards to comptime values and runtime values. In
addition, there was a hacky "mem_slot" mechanism that managed the memory
for local variables, and acted differently depending on comptime vs
runtime situations. All that is deleted in this commit, and as a result,
result locations code has one less complication.
Importantly, this means that a comptime result location is now passed to
a function when it is evaluated at comptime.
This test causes many regressions, and some of the behavior tests are
disabled (commented out) in this commit. Future commits will re-enable
the tests before merging the branch.
- `make` or `ninja` will not build but not install
- `make install` or `ninja install` will build __and__ install
Only for build system generator Visual Studio, specify the following
to disable installation of lib files:
ZIG_SKIP_INSTALL_LIB_FILES=ON
Fix breakpoints on macOS to trap EXC_BREAKPOINT with correct
source location when using lldb. Old behavior with `raise(SIGTRAP)`
traps SIGTRAP and incorrect source location.
Fix breakpoints on archlinux to trap SIGILL with correct source
location when using gdb. Old behavior with `raise(SIGTRAP)`
traps SIGTRAP and (sometimes) incorrect source location with
very shallow (break in main) stack.
when building stage1:
- w/ clang, use `__builtin_debugtrap()`
- w/ gcc, use `__builtin_trap()`
- else use `raise(SIGTRAP)`
Let's use the usual declaration-searching mechanism that resolves the
`usingnamespace` declarations on the go instead of directly peeking into
the symbol table.
Fixes#4134
Use a struct as second parameter to be future proof (and also allows to
specify default values for the parameters)
Closes#2679 as it was just a matter of a few lines of code.
Fixes#2379
= Overlong (non-shortest) sequences
UTF-8's unique encoding scheme allows for some Unicode codepoints
to be represented in multiple ways. For any of these characters,
the spec forbids all but the shortest form. These disallowed longer
sequences are called "overlong". As an interesting side effect of
this rule, the bytes C0 and C1 never appear in valid UTF-8.
= Codepoint range
UTF-8 disallows representation of codepoints beyond U+10FFFF,
which is the highest character which can be encoded in UTF-16.
Because a 4-byte sequence is capable of resulting in such characters,
they must be explicitly rejected. This rule also has an interesting
side effect, which is that bytes F5 to FF never appear.
= References
Detecting an overlong version of a codepoint could get gnarly, but
luckily The Unicode Consortium did the hard work by creating this
handy table of valid byte sequences:
https://unicode.org/versions/corrigendum1.html
I thought this mapped nicely to the parser's state machine, so I
rearranged the relevant states to make use of it.
* add reserved field to match musl struct definition so that
it will work with musl libc.
* add libc getrusage so that it will work with libc
what's not done in this branch is:
* test coverage. See #1629, which should also aim to provide
general test coverage for the std lib.
* rusage struct bits for non-linux operating systems
* `RunStep` moved to lib/std/build/run.zig and gains ability to compare
output and exit code against expected values. Multiple redundant
locations in the test harness code are replaced to use `RunStep`.
* `WriteFileStep` moved to lib/std/build/write_file.zig and gains
ability to write more than one file into the cache directory, for
when the files need to be relative to each other. This makes
usage of `WriteFileStep` no longer problematic when parallelizing
zig build.
* Added `CheckFileStep`, which can be used to validate that the output
of another step produced a valid file. Multiple redundant locations
in the test harness code are replaced to use `CheckFileStep`.
* Added `TranslateCStep`. This exposes `zig translate-c` to the build
system, which is likely to be rarely useful by most Zig users;
however Zig's own test suite uses it both for translate-c tests and
for run-translated-c tests.
* Refactored ad-hoc code to handle source files coming from multiple
kinds of sources, into `std.build.FileSource`.
* Added `std.build.Builder.addExecutableFromWriteFileStep`.
* Added `std.build.Builder.addExecutableSource`.
* Added `std.build.Builder.addWriteFiles`.
* Added `std.build.Builder.addTranslateC`.
* Added `std.build.LibExeObjStep.addCSourceFileSource`.
* Added `std.build.LibExeObjStep.addAssemblyFileFromWriteFileStep`.
* Added `std.build.LibExeObjStep.addAssemblyFileSource`.
* Exposed `std.fs.base64_encoder`.
Widening and truncating integer casting to different signedness
works better now. For example `(unsigned long)-1` is now translated
to zig code that compiles correctly.
* `std.mem.Compare` is now `std.math.Order` and the enum tags
renamed to follow new style convention.
* `std.mem.compare` is renamed to `std.mem.order`.
* new function `std.math.order`
Block-local identifiers have block-local mangling numbers, and more
consistent mangling is applied within blocks. Parameters, for example,
are treated the same as other block-local variables, and are not mangled
unless they conflict with another name in scope.
According to C11 6.5.6.8, pointer arithmetic may not overflow. In fact,
it may not even go more than 1 past the end of an object, or UB occurs.
This is the same as Zig pointer arithmetic semantics, and so the
`+` and `+=` operators rather than `+%` and `+%=` are appropriate for
C-translated pointer arithmetic.
* move test from std/io/test.zig to std/os/test.zig
* do glibc version check, and make direct system call if
glibc is too old
* disable test when not linking libc, to avoid not working
with outdated qemu version on the CI server. see #4019
In the code review I accidentally encouraged Luna to remove some
handling of errors that are possible according to POSIX, but I think how
Luna had it before was better, so I fixed it, and now the branch should
be good to merge.
Also faster, on my machine unicode/throughput_test.zig now gives e.g.
> original utf8ToUtf16Le: elapsed: 1048 ns (0 ms)
> new utf8ToUtf16Le: elapsed: 971 ns (0 ms)
* add runtime safety for slicing pointers, arrays, and slices.
* slicing without a sentinel value results in non-sentineled slice
* improved `std.debug.panic` handling of panic-during-panic
Not sure what the build platform is for the generated documentation,
and it's worth thinking about how best to deal with this pattern. It
might be worth figuring out how to rewrite this to have a single
definition of the public API with the implementation chosen at compile
time.
* #3844 update std.c functions to use null-terminated pointer types
* check linux functions
* fix callsites
* fix io test
* Add allocPrintCstr function to remove other cast
* always allow integer comparison operations no matter the
bit width, signedness, or comptime-ness of operands.
closes#2133
* implement comparisons for vectors, which returns vector of
bools.
closes#3001
This reverts commit b169f7b0d5.
This caused `integer cast truncated bits` at
std/child_process.zig:801:12
Can be reproduced on my machine simply by running `make`.
Previously, the compiler had special logic to determine whether to
include the startup code, which was in `std/special/start.zig`. Now,
the file is moved to `std/start.zig`, and there is no special logic
in the compiler. Instead, the standard library unconditionally imports
the `start.zig` file, which then has a `comptime` block that does the
logic of determining what, if any, start symbols to export. Instead of
`start.zig` being in its own special package, it is just another normal
file that is part of the standard library.
`std.builtin.TestFn` is now part of the standard library rather than
specially generated by the compiler.
This is not a meaningful abstraction. Use a for loop on the result
of `toSlice` or `toSliceConst`.
An iterator can be implemented on top of ArrayList by applications which
want additional functionality, such as removing elements while
iterating.
Closes#3037.
This change was mostly made with `zig fmt` and this also modified some whitespace. Note that in some files, `zig fmt` produced incorrect code, so the change was made manually.
Calling with a new stack, with a runtime-known stack pointer (e.g.
not a global variable) is regressed with this branch. It is now a
compile-error, due to the Runtime Hint system not being smart enough
to mix a compile-time modifier field with a runtime stack field.
I'm OK with this regression because this feature is flawed (see #3268)
and may be deleted from the language.
The new plan to support hobby operating systems is #3784.
And what kind of name is "Zen" anyway? There's already a
[Zen programming language](http://zenlang.sourceforge.net/)
and that's just confusing.
Total bytes used in stage1 std lib tests:
3.418 -> 3.198 GiB (saving 225 MiB)
There's still this from pass 1 not getting freed:
Const: 6909049 items, 72 bytes each, total 474.407 MiB
This is due to 2 things hanging on to references to IrAnalyze pointers:
* ZigVar->owner_exec->analysis
* LazyValue->ira
The LazyValue one could be solved by memoizing the results after the
lazy value is resolved, and then it could unref the IrAnalyze.
ZigVars that are determined to be comptime const, could have their
const_value set to that value, instead of using the mem_slot_index
mechanism. This would prevent an IrAnalyze ref in some cases.
Having ConstGlobalRefs be a pointer in ZigValue was a hack that caused
plenty of bugs. It was used to work around difficulties in type coercing
array values into slices.
However, after #3787 is merged, array values no longer type coerce into
slices, and so this provided an opportunity to clean up the code.
This has the nice effect of reducing stage1 peak RAM usage during the
std lib tests from 3.443 GiB to 3.405 GiB (saving 39 MiB).
There is one behavior test failing in this branch, which I plan to debug
after merging #3787.
All four of these can be represented in fewer characters with
expressions, which will be guaranteed to happen at compile-time, and
have the same or better precision.
The other math constants here which depend on function calls could be
similarly removed if and when #425 is solved. However I left them for
now since Zig does not eagerly evaluate functions with comptime parameters.
- on Windows use first found env var { "Path", "PATH" }
Bug Description: `build test` results in the following error on in
a msys64 shell with "PATH" env var instead of "Path":
error while loading shared libraries: mathtest.dll:
cannot open shared object file: No such file or directory
* Implements #3768. This is a sweeping breaking change that requires
many (trivial) edits to Zig source code. Array values no longer
coerced to slices; however one may use `&` to obtain a reference to
an array value, which may then be coerced to a slice.
* Adds `IrInstruction::dump`, for debugging purposes. It's useful to
call to inspect the instruction when debugging Zig IR.
* Fixes bugs with result location semantics. See the new behavior test
cases, and compile error test cases.
* Fixes bugs with `@typeInfo` not properly resolving const values.
* Behavior tests are passing but std lib tests are not yet. There
is more work to do before merging this branch.
The actual desired access mask in this case seems quite confusing and badly documented. The previous combination of `GENERIC_READ` and `SYNCHRONIZE` seems both illegal and redundant according to the [`ntifs.h` documentation](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-ntcreatefile), which specifies that `GENERIC_READ` should not be used for directories and includes `SYNCHRONIZE`. `winnt.h` contains a number of relevant-sounding flags such as `FILE_ADD_FILE`, `FILE_ADD_SUBDIRECTORY`, and `FILE_DELETE_CHILD` that do not show up in documentation at all. These are equal in value to file-specific flags that are documented as flags you should not specify when opening a directory.
The Windows-inspired nomenclature of "List" and "Traverse" was chosen over POSIX-style "Read" and "Path" (from `O_PATH`) for clarity. Using "Path" makes it look like the function is manipulating strings, and the generic "Read" ending isn't useful when there is no generic read method. Even in implementation details, `read` is never used.
Actual exploitation of the difference between the two functions will come in a later commit.
During build an empty .o on macOS/Xcode emits warning:
ranlib: file: zig_cpp/libcompiler.a(memory_profiling.cpp.o) has no symbols
ranlib: file: zig_cpp/libcompiler.a(memory_profiling.cpp.o) has no symbols
- extracted functions
- factorised extern "C" into a block containing all function prototypes instead of writing macros all over the place
- using intermediate buffers instead of writing directly to the output file
array_list.zig:
- adds ArrayList.initCapacity(*Allocator,usize) to allow preallocation of a block at initialization to reduce future allocations.
- adds a test for ArrayList.initCapacity() that ensures ArrayList.len() is unchanged and that at least the requested amount is allocated for.
buffer.zig:
- adds Buffer.initCapacity(*Allocator,usize), based off of ArrayList.initCapacity(), to preallocate a buffer before use.
note: contrary to Buffer.initSize(0) and then Buffer.list.ensureCapacity(200) (the presumed current method), this only allocates once instead of twice.
- adds Buffer.capacity to check usable allocated space, not including the null byte.
note: returns 0 when Buffer has only a null byte or when initNull() was used before without resize()/replaceContents().
- adds a test "Buffer.initCapacity" which ensures that Buffer.append()'s with [added size <= Buffer.capacity()-Buffer.len()] do not cause a reallocation to occur.
- adds a test "Buffer.initSize" which ensures that Buffer.initSize() behaves as expected, also combined with Buffer.append().
- adds a doc comment to Buffer.initSize() that makes its function and distinction from Buffer.initCapacity() clearer.
Added Buffer.initCapcity() to buffer to allow preallocation of a block of memory to reduce future allocations. Uses the added ArrayList.initCapacity() function to achieve this.
Added Buffer.capacity() to track current usable allocation size, not counting null byte, and returning 0 if empty or created with Buffer.initNull()
Added a test for initCapacity() that shows that no further allocation is performed for an append of size smaller than or equal to capacity when initCapacity is used.
Added a test for initSize(), since it did not exist already.
Also added a comment to better explain the difference between initSize() and initCapacity()
note: forgot in the first commit but thanks to mikdusan for helping me brainstorm, through the process, and for drawing up a draft diff which I tweaked.
Added ArrayList.initCapcity() as a way to preallocate a block of memory to reduce future allocations.
Added a test "std.ArrayList.initCapacity" that ensures initCapacity adds no elements and increases capacity by at least the requested amount
this also deletes C string literals from the language, and then makes
the std lib changes and compiler changes necessary to get the behavior
tests and std lib tests passing again.
Show differing pointer values when comparing pointers instead of the
content they point to.
It's confusing for a test to say "expected S{.x = 1}, found S{.x = 1}"
as illustrated below when it was the pointers that differed.
There seems to be different rules for when a pointer is dereferenced by
the printing routine depending on its type. I don't fully grok this but
it's also illustrated below.
const std = @import("std");
const S = struct { x: u32 };
// before: ...expected S{ .x = 1 }, found S{ .x = 1 }
// after: ...expected S@7ffcd20b7798, found S@7ffcd20b7790
test "compare_ptr_to_struct" {
var a = S{.x = 1};
var b = S{.x = 1};
std.testing.expectEqual(&a, &b);
}
// before: ...expected u32@7fff316ba31c, found u32@7fff316ba318
// after: ...expected u32@7ffecec622dc, found u32@7ffecec622d8
test "compare_ptr_to_scalar" {
var a: u32 = 1;
var b: u32 = 1;
std.testing.expectEqual(&a, &b);
}
This removes the remaining hack in the implementation of anonymous
struct literals, and they can now therefore now have greater than 16
fields/elements.
Thanks to the Windows Process Environment Block, it is possible to
obtain handles to the standard input, output, and error streams without
possibility of failure.
This implements stage1 parser support for anonymous struct literal
syntax (see #685), as well as semantic analysis support for anonymous
struct literals and anonymous list literals (see #208). The semantic
analysis works when there is a type coercion in the result location;
inferring the struct type based on the values in the literal is not
implemented yet. Also remaining to do is zig fmt support for this new
syntax and documentation updates.
This commit also hooks up type coercion (previously called implicit
casting) into the result location mechanism, and additionally hooks up
variable declarations, maintaining the property that:
var a: T = b;
is semantically equivalent to:
var a = @as(T, b);
See #1757
parseString() created a copy of the string using the wrong allocator.
Instead of using the ArenaAllocator, it was using the allocator passed
into Parser.init(). This lead to a leak as the copied string was not
freed when the ArenaAllocator was deinited.
This introduces the concept of vector index being part of a pointer
type. This avoids vectors having well-defined in-memory layout, and
allows vectors of any integer bit width to work the same way.
When a vector is indexed with a scalar, this is vector element access,
which this branch implements. When a vector is indexed with a vector,
this is gather/scatter, which is not implemented in this branch.
closes#3575closes#3580
Each of the 3 CI services now trigger a sr.ht build via
the on_master_success script. The sr.ht build checks if all builds have
completed successfully by trying to download the JSON file for the
particular version. If all builds have completed successfully then this
sr.ht job will update the download page.
- define zig global cache based on XDG spec:
if env XDG_CACHE_HOME {
"$XDG_CACHE_HOME/zig"
} else {
"$HOME/.cache/zig"
}
- old definition "$HOME/.local/share/zig" is retired
- closes#3573
* Delete `std.net.TmpWinAddr`. I don't think that was ever meant to
be a real thing.
* Delete `std.net.OsAddress`. This abstraction was not helpful.
* Rename `std.net.Address` to `std.net.IpAddress`. It is now an extern
union of IPv4 and IPv6 addresses.
* Move `std.net.parseIp4` and `std.net.parseIp6` to the
`std.net.IpAddress` namespace. They now return `IpAddress` instead of
`u32` and `std.net.Ip6Addr`, which is deleted.
* Add `std.net.IpAddress.parse` which accepts a port and parses either
an IPv4 or IPv6 address.
* Add `std.net.IpAddress.parseExpectingFamily` which additionally
accepts a `family` parameter.
* `std.net.IpAddress.initIp4` and `std.net.IpAddress.initIp6` are
improved to directly take the address fields instead of a weird
in-between type.
* `std.net.IpAddress.port` is renamed to `std.net.IpAddress.getPort`.
* Added `std.net.IpAddress.setPort`.
* `os.sockaddr` struct on all targets is improved to match the
corresponding system struct. Previously I had made it a union of
sockaddr_in, sockaddr_in6, and sockaddr_un. The new abstraction for
this is now `std.net.IpAddress`.
* `os.sockaddr` and related bits are added for Windows.
* `os.sockaddr` and related bits now have the `zero` fields default
to zero initialization, and `len` fields default to the correct size.
This is enough to abstract the differences across targets, and so
no more switch on the target OS is needed in `std.net.IpAddress`.
* Add the missing `os.sockaddr_un` on FreeBSD and NetBSD.
* `std.net.IpAddress.initPosix` now takes a pointer to `os.sockaddr`.
* delete the std/event/net directory
* `std.event.Loop.waitUntilFdReadable` and related functions
no longer have possibility of failure. On Linux, they fall
back to poll() and then fall back to sleep().
* add some missing `noasync` decorations in `std.event.Loop`
* redo the `std.net.Server` API. it's quite nice now, but
shutdown does not work cleanly. There is a race condition with
close() that I am actively working on.
* move `std.io.OutStream` to its own file to match `std.io.InStream`.
I started working on making `write` integrated with evented I/O,
but it got tricky so I backed off and filed #3557. However
I did integrate `std.os.writev` and `std.os.pwritev` with evented I/O.
* add `std.Target.stack_align`
* move networking tests to `lib/std/net/test.zig`
* add `std.net.tcpConnectToHost` and `std.net.tcpConnectToAddress`.
* rename `error.UnknownName` to `error.UnknownHostName` within the
context of DNS resolution.
* add `std.os.readv`, which is integrated with evented I/O.
* `std.os.preadv`, is now integrated with evented I/O.
* `std.os.accept4` now asserts that ENOTSOCK and EOPNOTSUPP never
occur (misuse of API), instead of returning errors.
* `std.os.connect` is now integrated with evented I/O.
`std.os.connect_async` is gone. Just use `std.os.connect`.
* fix false positive dependency loop regarding async function frames
* add more compile notes to help when dependency loops occur
in determining whether a function is async.
* ir: change an assert to ir_assert to make it easier to find
workarounds for when such an assert is triggered. In this case
it was trying to parse an IPv4 address at comptime.
d91fc0fdd8 changed zig's behavior to
disable the SSE feature when cross compiling for i386-freestanding.
This commit does the same when compiling C Code.
* Add some documentation for standard library things.
Added a bunch of descriptions for array_list.
Added some usage notes for failing_allocator.
Documented some of mem.Allocator.
I took a risk that docker had implemented namespacing with regards to
architectures correctly, but of course they didn't. So the CI docker
image for x86_64 was clobbered by an aarch64 image with the same name.
Now we're doing arch namespaces with the tag name, so here's the x86_64
one.
Clang does not support -march=native for all targets.
Arguably it should always work, but in reality it gives:
error: the clang compiler does not support '-march=native'
If we move CPU detection logic into Zig itelf, we will not need this,
instead we will always pass target features and CPU configuration explicitly.
For now, we simply avoid passing the flag when it is known to not be
supported.
It had the downside of running all the comptime blocks and resolving
all the usingnamespaces of each system, when just trying to discover if
the current system is a particular one.
For Darwin, where it's nice to use `std.Target.current.isDarwin()`, this
demonstrates the utility that #425 would provide.
* All the data types from `@import("builtin")` are moved to
`@import("std").builtin`. The target-related types are moved
to `std.Target`. This allows the data types to have methods, such as
`std.Target.current.isDarwin()`.
* `std.os.windows.subsystem` is moved to
`std.Target.current.subsystem`.
* Remove the concept of the panic package from the compiler
implementation. Instead, `std.builtin.panic` is always the panic
function. It checks for `@hasDecl(@import("root"), "panic")`,
or else provides a default implementation.
This is an important step for multibuilds (#3028). Without this change,
the types inside the builtin namespace look like different types, when
trying to merge builds with different target settings. With this change,
Zig can figure out that, e.g., `std.builtin.Os` (the enum type) from one
compilation and `std.builtin.Os` from another compilation are the same
type, even if the target OS value differs.
The sockaddr pointer and size of the accept function points to a data structure that can only be determined at runtime. The only requirement is that it must be large enough to hold 2 bytes for the address family value. Typeical usage of the socket API is for UDP/TCP IPv4 and IPv6 sockets, which use sockaddr_in and sockaddr_in6. And some sockets can actually support both simultaneously in which case the app may want to have access to the size of the returned sockaddr. Operating systems can even support custom protocols where they use custom sockaddr data structures. In this case the standard library would have no knowledge of the actual size of the sockaddr being passed into the accept function. In this case the standard library should defer to the app to pass in the size of their structure.
* Added `std.c.unlinkat` and `std.os.unlinkat`.
* Removed `std.fs.MAX_BUF_BYTES` (this declaration never made it to
master branch)
* Added `std.fs.Dir.deleteTree` to be used on an open directory handle.
* `std.fs.deleteTree` has better behavior for both relative and
absolute paths. For absolute paths, it opens the base directory
and uses that handle for subsequent operations. For relative paths,
it does a similar strategy, using the cwd handle.
* The error set of `std.fs.deleteTree` is improved to no longer have
these possible errors:
- OutOfMemory
- FileTooBig
- IsDir
- DirNotEmpty
- PathAlreadyExists
- NoSpaceLeft
* Added `std.fs.Dir.posix_cwd` which is a statically initialized
directory representing the current working directory.
* The error set of `std.Dir.open` is improved to no longer have these
possible errors:
- FileTooBig
- IsDir
- NoSpaceLeft
- PathAlreadyExists
- OutOfMemory
* Added more alternative functions to `std.fs` for when the path
parameter is a null terminated string. This can sometimes be more
effecient on systems which have an ABI based on null terminated
strings.
* Added `std.fs.Dir.openDir`, `std.fs.Dir.deleteFile`, and
`std.fs.Dir.deleteDir` which all operate on an open directory handle.
* `std.fs.Walker.Entry` now has a `dir` field, which can be used to do
operations directly on `std.fs.Walker.Entry.basename`, avoiding
`error.NameTooLong` for deeply nested paths.
* Added more docs to `std.os.OpenError`
This commit does the POSIX components for these changes. I plan to
follow up shortly with a commit for Windows.
This brings the std lib tests down from 3.51 GiB memory usage
to 3.41 GiB, by making two fields that were 64 bits 32 bits.
This is a small thing; the bigger wins will come from the strategy
outlined in the previous commit.
* use erase rest of line escape code.
* use `stderr.supportsAnsiEscapeCodes` rather than `isTty`.
* respect `--color off`
* avoid unnecessary recursion
* add `Progress.log`
* disable the progress std lib test since it's noisy and uses
`time.sleep()`.
* enable/integrate progress printing with the default test runner
* `std.os.execve` had the wrong name; it should have been
`std.os.execvpe`. This is now corrected.
* introduce `std.os.execveC` which does not look at PATH, and uses
null terminated parameters, matching POSIX ABIs. It does not
require an allocator.
* fix typo nonsense doc comment in `std.fs.MAX_PATH_BYTES`.
* introduce `std.os.execvpeC`, which is like `execvpe` except it
uses null terminated parameters, matching POSIX ABIs, and thus
does not require an allocator.
* `std.os.execvpe` implementation is reworked to only convert
parameters and then delegate to `std.os.execvpeC`.
* `std.os.execvpeC` improved to handle `ENOTDIR`. See #3415
previously zig would look for all the .defs even when not needed,
wasting time. also, we only had samsrv definitions for some architectures.
remove cross compiling support for this.
Some keyboard layouts produces a different ev.which value in firefox
for ? than 191, eg. the Swedish QWERTY one produces 171. Chrome/chromium
doesn't have this issue.
* introduce std.json.WriteStream API for writing json
data to a stream
* add WIP tools/merge_anal_dumps.zig for merging multiple semantic
analysis dumps into one. See #3028
* add std.json.Array, improves generated docs
* add test for `std.process.argsAlloc`, improves test coverage and
generated docs
`ir_resolve_str()` bug returns array expression even when when sliced
to a lesser length. Fix is to return array if slice.len == array.len,
otherwise return slice.
Bug report use-case is based on one builtin function. However, at least
the following builtins were exposed to the bug:
`@byteOffsetOf`
`@cDefine`
`@cImport`
`@cInclude`
`@cUndef`
`@compileError`
`@embedFile`
`@export`
`@fieldParentPtr`
`@hasDecl`
`@hasField`
`@import`
`@unionInit`
closes#3384
And then get the struct field astNodes through the containerDecl astNode.
The type of a struct field is still stored in the types array, but the
static information is in the astNodes.
Lift some code from llvm-dlltool, the lld code is meant to follow what
gnu ld does but that's not much useful for our purposes.
Also use the `--kill-at` option when generating the .lib files out of
mingw's .def files: this way our building process closely matches the
one use by the upstream and now finally generates files that allow both
C code and Zig code to link.
- during diagnostics the string representation for root was empty
and now is `(root)`
- retrofitted all other namespace-qualified type naming to elide
prefixing with root
closes#2032
Clang pretends to be gcc 4.4 and that causes some re-definition errors.
The problem has been reported to the upstream some time ago but nothing
was done about it.
we don't really have a way to determine whether the stage1
zig compiler requires PIC so to be safe we always enable it
when building libuserland.
fixes build on some configurations of alpine linux.
- decls brought in via `usingnamespace` were not always found
because lookup was performed directly against decl_table and
use_decls was never consulted
- fix to use find_container_decl() path instead
- closes#3367
This commit adds `-fgenerate-docs` CLI option, and it outputs:
* doc/index.html
* doc/data.js
* doc/main.js
In this strategy, we have 1 static html page and 1 static javascript
file, which loads the semantic analysis dump directly and renders it
using dom manipulation.
Currently, all it does is list the declarations. But there is a lot more
data available to work with. The next step would be making the
declarations hyperlinks, and handling page navigation.
Another strategy would be to generate a static site with no javascript,
based on the semantic analysis dump that zig now provides. I invite the
Zig community to take on such a project. However this version which
heavily relies on javascript will also be a direction explored.
I also welcome contributors to improve the html, css, and javascript of
what this commit started, as well as whatever improvements are necessary
to the static analysis dumping code to provide more information.
See #21.
This commit adds -fdump-analysis which creates
a `$NAME-analysis.json` file with all of the finished
semantic analysis that the stage1 compiler produced.
It contains types, packages, declarations, and files.
This is an initial implementation; some data will be
missing. However it's easy to improve the implementation,
which is in `src/dump_analysis.cpp`.
The next step for #21 will be to create Zig code which parses
this json file and creates user-facing HTML documentation.
This feature has other uses, however; for example, it could
be used for IDE integration features until the self-hosted
compiler is available.
See #1964
translate_c.cpp now exclusively uses the clang API via zig_clang.h
shaves off 5 seconds from building zig when translate_c.cpp
(or any h files it uses) change.
* [download a pre-built binary](https://ziglang.org/download/)
* [install from a package manager](https://github.com/ziglang/zig/wiki/Install-Zig-from-a-Package-Manager)
* [build from source](https://github.com/ziglang/zig/wiki/Building-Zig-From-Source)
* [bootstrap zig for any target](https://github.com/ziglang/zig-bootstrap)
Note that you can
[download a binary of master branch](https://ziglang.org/download/#release-master).
## License
### Stage 1: Build Zig from C++ Source Code
The ultimate goal of the Zig project is to serve users. As a first-order
effect, this means users of the compiler, helping programmers to write better
software. Even more important, however, are the end-users.
#### Dependencies
Zig is intended to be used to help **end-users** accomplish their goals. Zig
should be used to empower end-users, never to exploit them financially, or to
limit their freedom to interact with hardware or software in any way.
##### POSIX
However, such problems are best solved with social norms, not with software
licenses. Any attempt to complicate the software license of Zig would risk
compromising the value Zig provides.
* cmake >= 2.8.5
* gcc >= 5.0.0 or clang >= 3.6.0
* LLVM, Clang, LLD development libraries == 9.x, compiled with the same gcc or clang version above
- Use the system package manager, or [build from source](https://github.com/ziglang/zig/wiki/How-to-build-LLVM,-libclang,-and-liblld-from-source#posix).
Therefore, Zig is available under the MIT (Expat) License, and comes with a
humble request: use it to make software better serve the needs of end-users.
##### Windows
* cmake >= 3.15.3
* Microsoft Visual Studio. Supported versions:
- 2015 (version 14)
- 2017 (version 15.8)
- 2019 (version 16)
* LLVM, Clang, LLD development libraries == 9.x
- Use the [pre-built binaries](https://github.com/ziglang/zig/wiki/Building-Zig-on-Windows) or [build from source](https://github.com/ziglang/zig/wiki/How-to-build-LLVM,-libclang,-and-liblld-from-source#windows).
message(FATAL_ERROR"LLVM (according to ${LLVM_CONFIG_EXE}) is missing target ${TARGET_NAME}. Zig requires LLVM to be built with all default targets enabled.")
"Resolves symbols in any (transitively) linked dynamic libraries. Source libraries are not recorded: dyld will re-search all images at runtime and use the first definition found.",nullptr,nullptr)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
