This reverts commit 06310e3d4e, reapplying
commit a430630002.
I deeply apologize to @moosichu and those affected by this bug. The
original fix was actually fine. When I reverted it, I misremembered
how the Cache API works. I thought the fix was going to introduce
nondeterminism into the hash, but I forgot that the order of files in
the manifest doesn't actually matter when checking for a cache hit.
Actually, it does matter a little bit. This fix has a subtle downside
which is that it does introduce the possibility of false negatives when
checking for cache hits of 2+ iterations ago. For example, if the code
goes from "foo", to "bar", and then back to "foo", it may look like a
cache miss when it should have been a hit because 2 iterations ago the
code was the same. However, this is an uncommon use case, and all it
does is cause a bit of wasted time and disk space. That said, my
suggestion from earlier still applies and would be a nice follow-up
enhancement to this fix:
The proper solution to this is to, in whole cache mode, append the hash
inputs to some data structure, and then after the compilation is
complete, do some kind of sorting on the hash inputs so that they will
be the same order every time, then apply them in sequence. No lock on
the Cache object is needed for this scheme.
closes#11063
Instead of making the memory alignment functions more complicated, I
added more API documentation for their existing semantics.
closes#12118closes#12135
Previously, if you used `zig test -ofmt=c -target foobar` then Zig would
try to compile the generated C code with the native target instead of
"foobar".
With this change, `--test-cmd` with e.g. QEMU still won't work, but at
least the binary will get compiled for the correct target.
We need to be careful to respect side-effects/branching in these
cases, but otherwise this behaves very similarly to multiplication.
`lhs and rhs == false` if either lhs or rhs is comptime-known `false`,
just like `lhs * rhs == 0` if either lhs or rhs is comptime-known to
be zero.
Similar reasoning applies to `lhs or rhs`.
* std.os.uefi: integer backed structs, add tests to catch regressions
device_path_protocol now uses extern structs with align(1) fields because
the transition to integer backed packed struct broke alignment
added comptime asserts that device_path_protocol structs do not violate
alignment and size specifications
this is a hack meant to restore functionality for the upcoming release,
a proper analysis of the new zir structure is required to make a robust
change.
Make the test use the minimum length and set MAX_NAME_BYTES to the maximum so that:
- the test will work on any host platform
- *and* the MAX_NAME_BYTES will be able to hold the max file name component on any host platform
Each u16 within a file name component can be encoded as up to 3 UTF-8 bytes, so we need to use MAX_NAME_BYTES to account for all possible UTF-8 encoded names.
Fixes#8268
Before, the code for building glibc stubs used a special case of the
Cache API that did not add any file inputs, and did not use
writeManifest(). This is not really how the Cache API is designed to
work and it shows because there was a race condition.
This commit adds as an input file the abilists file that comes with
Zig's installation, which has the added benefit of making glibc stub
caching properly detect cache invalidation when the user decides to
overwrite their abilists file. This harmonizes with the rest of how Zig
works, which intentionally allows you to hack the installation files and
have it behave properly with the cache system.
Finally, because of having any file inputs, the normal API flow of the
Cache system can be used, eliminating the one place that used the Cache
API in a special way. In other words, it uses writeManifest() now and
properly obeys the cache hit/miss semantics.
closes#13160
Comptime code can't execute assembly code, so we need some way to
force comptime code to use the generic path. This should be replaced
with whatever is implemented for #868, when that day comes.
I am seeing that the result for the hash is incorrect in stage1 and
crashes stage2, so presumably this never worked correctly. I will follow
up on that soon.
This gets us most of the way back to the performance I had when
I was using the LLVM intrinsics:
- Intel Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz:
190.67 MB/s (w/o intrinsics) -> 1285.08 MB/s
- AMD EPYC 7763 (VM) @ 2.45 GHz:
240.09 MB/s (w/o intrinsics) -> 1360.78 MB/s
- Apple M1:
216.96 MB/s (w/o intrinsics) -> 2133.69 MB/s
Minor changes to this source can swing performance from 400 MB/s to
1400 MB/s or... 20 MB/s, depending on how it interacts with the
optimizer. I have a sneaking suspicion that despite LLVM inheriting
GCC's extremely strict inline assembly semantics, its passes are
rather skittish around inline assembly (and almost certainly, its
instruction cost models can assume nothing)
There's probably plenty of room to optimize these further in the
future, but for the moment this gives ~3x improvement on Intel
x86-64 processors, ~5x on AMD, and ~10x on M1 Macs.
These extensions are very new - Most processors prior to 2020 do
not support them.
AVX-512 is a slightly older alternative that we could use on Intel
for a much bigger performance bump, but it's been fused off on
Intel's latest hybrid architectures and it relies on computing
independent SHA hashes in parallel. In contrast, these SHA intrinsics
provide the usual single-threaded, single-stream interface, and should
continue working on new processors.
AArch64 also has SHA-512 intrinsics that we could take advantage
of in the future
Packed memory has a well-defined layout that doesn't require
conversion from an integer to read from. Let's use it :-)
This change means that for bitcasting to/from a packed value that
is N layers deep, we no longer have to create N temporary big-ints
and perform N copies.
Other miscellaneous improvements:
- Adds support for casting to packed enums and vectors
- Fixes bitcasting to/from vectors outside of a packed struct
- Adds a fast path for bitcasting <= u/i64
- Fixes bug when bitcasting f80 which would clear following fields
This also changes the bitcast memory layout of exotic integers on
big-endian systems to match what's empirically observed on our targets.
Technically, this layout is not guaranteed by LLVM so we should probably
ban bitcasts that reveal these padding bits, but for now this is an
improvement.
Similar to what was done for EdDSA, allow incremental creation
and verification of ECDSA signatures.
Doing so for ECDSA is trivial, and can be useful for TLS as well
as the future package manager.
* Old cmake option: `-DZIG_SKIP_INSTALL_LIB_FILES=ON`
* New cmake option: `-DZIG_NO_LIB=ON`
* Old build.zig option: `-Dskip-install-lib-files`
* New build.zig option: `-Dno-lib`
Motivation is making build commands easier to type.
This definition communicates to libcxxabi that the libc will provide the
`__cxa_thread_atexit_impl` symbol. This is true for glibc but not
true for other libcs, such as musl.
This commit accepts unusual parameters like EcdsaP384Sha256.
Some certifictes(below certs are in /etc/ssl/certs/ca-certificates.crt on Ubuntu 22.04) use EcdsaP384Sha256 to sign itself.
- Subject: C=GR, L=Athens, O=Hellenic Academic and Research Institutions Cert. Authority, CN=Hellenic Academic and Research Institutions ECC RootCA 2015
- Subject: C=US, ST=Texas, L=Houston, O=SSL Corporation, CN=SSL.com EV Root Certification Authority ECC
- Subject: C=US, ST=Texas, L=Houston, O=SSL Corporation, CN=SSL.com Root Certification Authority ECC
In verify(), hash array `h` is allocated to be larger than the scalar.encoded_length.
The array is regarded as big-endian.
Hash values are filled in the back of the array and the rest bytes in front are filled with zero.
In sign(), the hash array is allocated and filled as same as verify().
In deterministicScalar(), hash bytes are insufficient to generate `k`
To generate `k` without narrowing its value range,
this commit uses algorithm stage h. in "Section 3.2 Generation of k" in RFC6979.
Considering all possible features are known by the linker during
compile-time, we can create arrays on the stack instead of
dynamically allocating hash maps. We use a simple bitset to determine
whether a feature is enabled or not, and from which object file
it originates. This allows us to make feature validation slightly
faster and use less runtime memory.
In the future this could be enhanced further by having a single
array instead with a more sophisticated bitset.
The list of features a Wasm object/binary file can emit can differ
from the list of cpu features. The reason for this is because the
"target_features" section also contains linker features. An example
of this is the "shared-mem" feature, which is a feature for the linker
and not that of the cpu target as defined by LLVM.
Adds a test for inferring features based on a different object file.
Also provides a test case where cpu features are explicitly set on
a library where the end result will output said target features.
When an object file or binary contains the target_features section
we can now parse and then dump its contents in string format so
we can use them in our linker tests to verify the features section.
When the result is not being stripped, we emit the `target_features`
section based on all the used features. This includes features
inferred from linked object files.
Considering we know all possible features upfront, we can use an
array and therefore do not have to dynamically allocate memory.
Using this trick we can also easily order all features based
the same ordering as found in `std.Target.wasm` which is the same
ordering used by LLVM and the like.
Verifies disallowed and used/required features. After verifying,
all errors will be emit to notify the user about incompatible
features. When the user did not define any featureset, we infer
the features from the linked objects instead.
These ifs were missing a case for f80 which should have shifted by one,
but we can just compute the correct value instead. Also, we want the
fractional bits to be a multiple of four, not the mantissa bits, since
the mantissa could have a leading one which we want to be separated.
Global constant initializers can reference functions, so forward declare
the constants and initialize them later with the function definitions,
which guarantees that they appear after all declarations.
These ifs were missing a case for f80 which should have shifted by one,
but we can just compute the correct value instead. Also, we want the
fractional bits to be a multiple of four, not the mantissa bits, since
the mantissa could have a leading one which we want to be separated.
This makes it easier to understand how control flow should happen in
various cases; already just by doing this it is revealed that
UndefinedSymbol and UndefinedSymbolReference should be merged, and that
MissingMainEntrypoint should be removed in favor of the ErrorFlags
mechanism thath we already have for missing the main entrypoint.
The main motivation for this change, however, is preventing a compile
error when there is conditional compilation inside linker
implementations, causing the flush() error set to depend on compilation
options. With this change, the error set is fixed, and, notably, the
`-Donly-c` flag no longer has compilation errors due to this error set.
The larger alignment on this platform means that long double reports
a sizeof 16 bytes, but it's underlying size is really just the 10
bytes of `f80`
C doesn't give us a way to see the "underlying" size of a type, so
this has to be caught by hand or by monitoring runtime memory. Luckily,
x86 and x86-64 are the only platforms that seem to use a non-power-of-two
type like this.
This option can be used to produce a C backend build of the self-hosted
compiler, which only has the C backend enabled. Once the C backend is
capable of self-hosting, this will be a way for us to replace our stage1
codebase with a C backend build of self-hosted, which we can then use
for bootstrapping. See #5246 for more details.
Using this option right now results in a crash because the C backend is
not yet passing all the behavior tests.
CMake recognizes the CMAKE_PREFIX_PATH environment variable for some
things, and also the CMAKE_PREFIX_PATH cache variable for other things.
However, it does not relate these two things, i.e. if the environment
variable is set, CMake does not populate the cache variable in a
corresponding manner. Some package systems, such as Homebrew, set the
environment variable but not the cache variable. Furthermore, the
environment variable follows the system path separator, such as ':' on
POSIX and ';' on Windows, but the cache variable follows CMake's array
behavior, i.e. always ';' for a separator.
Closes#13242
This value corresponds to clang/gcc's `__alignof` (rather than
`_Alignof` which reports the minimum alignment). We don't use this
information yet, but it might be useful for implementing ABIs so it
is included here.
The Zig LLVM backend emits calls to softfloat methods with the "standard
compiler-rt" names. Rather than add complexity to the backend and
have to synchronize the naming scheme across all targets, the simplest
fix is just to export these symbols under both the "standard" and the
platform-specific naming convention.
This documents status of routines and adds the next work item
"Decimal float library routines", which are only recommended for
binary data. Complete absence of tests is also documented.
This does not document the various aliases, e.g. those for ARM.
Missing Integer library routines:
- __addvsi3
- __addvdi3
- __addvti3
- __addvdi3
- __addvti3
- __subvsi3
- __subvdi3
- __subvti3
- __subvdi3
- __subvti3
- __mulvsi3
- __mulvdi3
- __mulvti3
- __mulvdi3
- __mulvti3
Missing floating library routines:
- __powisf2
- __powidf2
- __powitf2
- __powixf2
Missing routines for symbol-level compatibility to gcc:
- __ashlsi3
- __ashrsi3
- __lshrsi3
cmp.zig was accidently being referenced twice, rather than importing
memcmp.zig. This means that its symbols were also not included in
the generated compiler-rt output.
1. If an object file was not compiled with `MH_SUBSECTIONS_VIA_SYMBOLS`
such a hand-written ASM on x86_64, treat the entire object file as
not suitable for dead code stripping aka a GC root.
2. If there are non-extern relocs within a section, treat the entire
section as a root, at least temporarily until we work out the exact
conditions for marking the atoms live.
This function is redundant with CType.sizeInBits(), and until the
previous commit they disagreed about the correct long double type
for several targets. Although they're all synced up now, it's much
simpler just to have a single source of truth.
These updates were made by testing against the `sizeof/_Alignof` reported
by Clang for all supported arch-OS-ABI combinations and correcting any
discrepancies.
This is bound to have a few errors (the recent long double fix for i386
Android is one example), but Clang is certainly not a bad place to start,
especially for our most popular targets.
Instead of adding 3 fields to every `Block`, this adds just one. The
function-level information is saved in the `Sema` struct instead,
which is created/copied more rarely.
Previously, we'd overwrite the errors in a circular buffer. Now that
error return traces are intended to follow a stack discipline, we no
longer have to support the index rolling over. By treating the trace
like a saturating stack, any pop/restore code still behaves correctly
past-the-end of the trace.
As a bonus, this adds a small blurb to let the user know when the trace
saturated and x number of frames were dropped.
This change extends the "lifetime" of the error return trace associated
with an error to continue throughout the block of a `const` variable
that it is assigned to.
This is necessary to support patterns like this one in test_runner.zig:
```zig
const result = foo();
if (result) |_| {
// ... success logic
} else |err| {
// `foo()` should be included in the error trace here
return error.TestFailed;
}
```
To make this happen, the majority of the error return trace popping logic
needed to move into Sema, since `const x = foo();` cannot be examined
syntactically to determine whether it modifies the error return trace. We
also have to make sure not to delete pertinent block information before it
makes it to Sema, so that Sema can pop/restore around blocks correctly.
* Why do this only for `const` and not `var`? *
There is room to relax things for `var`, but only a little bit. We could
do the same thing we do for const and keep the error trace alive for the
remainder of the block where the *assignment* happens. Any wider scope
would violate the stack discipline for traces, so it's not viable.
In the end, I decided the most consistent behavior for the user is just
to kill all error return traces assigned to a mutable `var`.
Despite the old doc-comment, this function cannot be valid for all types
since it operates with only a value and Error (Union) types have
overlapping Value representations with other Types.
This change extends the "lifetime" of the error return trace associated
with an error to include the duration of a function call it is passed
to.
This means that if a function returns an error, its return trace will
include the error return trace for any error inputs. This is needed to
support `testing.expectError` and similar functions.
If a function returns a non-error, we have to clean up any error return
traces created by error-able call arguments.
This re-factor is intended to make it easier to track what kind of
operator/expression consumes a result location, without overloading the
ResultLoc union for this purpose.
This is used in the following commit to keep track of initializer
expressions of `const` variables to avoid popping error traces
pre-maturely. Hopefully this will also be useful for implementing
RLS temporaries in the future.
This is encoded as a primitive AIR instruction to resolve one corner
case: A function may include a `catch { ... }` or `else |err| { ... }`
block but not call any errorable fn. In that case, there is no error
return trace to save the index of and codegen needs to avoid
interacting with the non-existing error trace.
By using a primitive AIR op, we can depend on Liveness to mark this
unused in this corner case.
In order to enforce a strict stack discipline for error return traces,
we cannot track error return traces that are stored in variables:
```zig
const x = errorable(); // errorable()'s error return trace is killed here
// v-- error trace starts here instead
return x catch error.UnknownError;
```
In order to propagate error return traces, function calls need to be passed
directly to an error-handling expression (`if`, `catch`, `try` or `return`):
```zig
// When passed directly to `catch`, the return trace is propagated
return errorable() catch error.UnknownError;
// Using a break also works
return blk: {
// code here
break :blk errorable();
} catch error.UnknownError;
```
Why do we need this restriction? Without it, multiple errors can co-exist
with their own error traces. Handling that situation correctly means either:
a. Dynamically allocating trace memory and tracking lifetimes, OR
b. Allowing the production of one error to interfere with the trace of another
(which is the current status quo)
This is piece (3/3) of https://github.com/ziglang/zig/issues/1923#issuecomment-1218495574
This allows for errors to be "re-thrown" by yielding any error as the
result of a catch block. For example:
```zig
fn errorable() !void {
return error.FallingOutOfPlane;
}
fn foo(have_parachute: bool) !void {
return errorable() catch |err| b: {
if (have_parachute) {
// error trace will include the call to errorable()
break :b error.NoParachute;
} else {
return;
}
};
}
pub fn main() !void {
// Anything that returns a non-error does not pollute the error trace.
try foo(true);
// This error trace will still include errorable(), whose error was "re-thrown" by foo()
try foo(false);
}
```
This is piece (2/3) of https://github.com/ziglang/zig/issues/1923#issuecomment-1218495574
This implement trace "popping" for correctly handled errors within
`catch { ... }` and `else { ... }` blocks.
When breaking from these blocks with any non-error, we pop the error
trace frames corresponding to the operand. When breaking with an error,
we preserve the frames so that error traces "chain" together as usual.
```zig
fn foo(cond1: bool, cond2: bool) !void {
bar() catch {
if (cond1) {
// If baz() result is a non-error, pop the error trace frames from bar()
// If baz() result is an error, leave the bar() frames on the error trace
return baz();
} else if (cond2) {
// If we break/return an error, then leave the error frames from bar() on the error trace
return error.Foo;
}
};
// An error returned from here does not include bar()'s error frames in the trace
return error.Bar;
}
```
Notice that if foo() does not return an error it, it leaves no extra
frames on the error trace.
This is piece (1/3) of https://github.com/ziglang/zig/issues/1923#issuecomment-1218495574
This makes possible to query the memory map size from EFI firmware
without making any allocation beforehand. This makes possible to be
precise about the size of the allocation which will own a copy of
the memory map from the UEFI application.
It is possible to get comptime-known values from runtime-known values
for example the length of array. Allowing runtime only instructions to
be emitted outside function bodies allows these operations to happen.
In places where comptime-known values are required we have other methods
to ensure that and they usually result in more specific compile errors too.
Closes#12240
I have noticed this causing my terminal to stop accepting input
sometimes. The previous implementation with all of its flaws was better
in the sense that it never caused this to happen.
This commit has multiple reverts in it:
Revert "Merge pull request #13148 from r00ster91/progressfollowup"
This reverts commit cb257d59f9, reversing
changes made to f5f28e0d2c.
Revert "`std.Progress`: fix inaccurate line truncation and use optimal
max terminal width (#12079)"
This reverts commit cd3d8f3a4e.
Windows gives AccessDenied if you delete a directory which contains open
file handles. This could be triggered when using CacheMode.whole when
cross compiling macho test binaries.
build.zig: add a 'compile' step to compile the self-hosted compiler
without installing it.
Compilation: set cache mode to whole when using the LLVM backend and
--enable-cache is passed.
This makes `zig build` act the same as it does with stage1. Upside is
that a second invocation of `zig build` on an unmodified source tree
will avoid redoing the compilation again. Downside is that it will
proliferate more garbage in the project-local cache (same as stage1).
This can eventually be fixed when Zig's incremental compilation is more
robust; we can go back to having LLVM use CacheMode.incremental and rely
on it detecting no changes and avoiding doing the flush() step.
enums so that we can branch to set `link_mode` properly when we iterate
over the clang arguments. also replaced `dynamic` flag in
clang_options_data.zig with proper definition similarly to `static`."
This reverts commit 6af0eeb58d.
This change needs more careful consideration. It regressed
zig-bootstrap due to cmake passing `-static -lkernel32` and zig failing
with error.UnableToStaticLink.
See https://github.com/ziglang/zig-bootstrap/issues/134
Currenty copy_file_range always uses at least two syscalls:
1. As many as it needs to do the initial copy (always 1 during my
testing)
2. The last one is always when offset is the size of the file.
The second syscall is used to detect the terminating condition. However,
because we do a stat for other reasons, we know the size of the file,
and we can skip the syscall.
Sparse files: since copy_file_range expands holes of sparse files, I
conclude that this layer was not intended to work with sparse files. In
other words, this commit does not make it worse for sparse file society.
Test program
------------
const std = @import("std");
pub fn main() !void {
const arg1 = std.mem.span(std.os.argv[1]);
const arg2 = std.mem.span(std.os.argv[2]);
try std.fs.cwd().copyFile(arg1, std.fs.cwd(), arg2, .{});
}
Test output (current master)
----------------------------
Observe two `copy_file_range` syscalls: one with 209 bytes, one with
zero:
$ zig build-exe cp.zig
$ strace ./cp ./cp.zig ./cp2.zig |& grep copy_file_range
copy_file_range(3, [0], 5, [0], 4294967295, 0) = 209
copy_file_range(3, [209], 5, [209], 4294967295, 0) = 0
$
Test output (this diff)
-----------------------
Observe a single `copy_file_range` syscall with 209 bytes:
$ /code/zig/build/zig build-exe cp.zig
$ strace ./cp ./cp.zig ./cp2.zig |& grep copy_file_range
copy_file_range(3, [0], 5, [0], 4294967295, 0) = 209
$
I did not fully wire it up in main.zig when I originally implemented
`-z nocopyreloc` in #11679 (440f5249f1a). Finish it.
If we strictly follow the rules, we should bump the cache has version,
since the field was technically added only now. But since nobody
complained thus far, I don't think many users care that much about it
and we can omit it.
When creating an `EmulatableRunStep`, it now correctly depends
on its own step rather than only the executable that was created.
This means we do not need to add extra `dependOn` statements on
both the emulatable step as well as the check object step.
same change as [68e26a2cee] "std: check for overflow in writeCurrentStackTrace"
On arm64 macOS, the address of the last frame is 0x0 rather than
a positive value like 0x1 on x86_64 macOS, therefore, we overflow
an integer trying to subtract 1 when printing the stack trace. This
patch fixes it by first checking for this condition before trying
to subtract 1.
Same behaviour on i386-windows-msvc.
Note that we do not need to signal the `SignalIterator` about this
as it will correctly detect this condition on the subsequent iteration
and return `null`, thus terminating the loop.
* Sema: implement linksection on functions
* Implement function linksection in Sema.
* Don't clobber function linksection/align/addrspace in Sema.
* Fix copy-paste typo in tests.
* Add a bunch of missing test_step.dependOn.
* Fix checkInSymtab match.
Closes#12546
* Fix for: DefaultRwLock accumulates write-waiters, eventually fails to write lock #13163
* Comment out debug.print at the end of the last test.
* Code formatting
* - use equality test after lock/unlock rather than peeking into internals.
however, this is still implementation specific and only done for
DefaultRwLock.
- add num_reads maximum to ensure that reader threads stop if writer threads are
starved
- use relaxed orderings for the read atomic counter
- don't check at the end for non-zero read ops, since the reader threads may
only run once if they are starved
* More review changes
- Monotonic is sufficient for incrementing the reads counter
It is not yet determined whether the Zig language will land on
text-based string concatenation for inline assembly, as Zig 0.9.1
allows, and as this commit allows, or whether it will introduce a new
assembly syntax more integrated with the rest of the language. Until
this decision is made, this commit relaxes the restriction which was
preventing inline assembly expressions from using comptime expressions
for the assembly source code.
Commit f14cc75 accidentally added a const when grepping for assignments
to `std.builtin.Type.StructField.default_value`, however when looking
into it further, I noticed that even though this default_value field is
emitted into the .data section, the value it points to is actually
emitted into the .rodata section, so it seems correct to use const here.
- For ALU operations, src should be allowed to be an explicit Reg.
- Expose AluOp and JmpOp as public types.
This makes code generation using BPF as a backend easier,
as AluOp and JmpOp can be used directly as part of an IR
'Self' isn't a very good name to describe what it does.
This commit changes the type name into `CodeGen` and the parameter
to `func` as we're generating code for a function.
With this change, the backend's coding style is in line with the
self-hosted Wasm-linker.
When we return an operand directly as a result, we must call
`reuseOperand`. This commit ensures it's done for all currently-
implemented AIR instructions.
Rather than accepting a canonical branch and a target branch
we allow to directly merge a branch into the parent branch.
This is possible as there's no overlapping and we have infinite
registers to our availability. This makes merging a lot simpler.
When determining the type of a local (read: register), we would
previously subtract the stack locals also. However, this locals
are also within the same `locals` list, meaning the type of the
local we were retrieving was off by 2. This could create a validation
error when we re-use a local of a different type.
Upon a branch, we only allow locals to be freed which were allocated
within the same branch as where they die. This ensures that when two
or more branches target the same operand we do not try to free
it more than once. This does however not implement freeing the local
upon branch merging yet.
When reusing an operand it increases the reference count, then when
an operand dies it will only decrease the reference count. When
this reaches 0, the local will be virtually freed, meaning it can be
re-used for a new local.
By reference counting the locals, we can ensure that when we free
a local, no local will be reused while it still has references pointing
to it. This prevents misscompilations. The compiler will also panic if
we free a local more than we reference it, introducing extra safety to
ensure they match up.
This hooks reusal of locals into liveness analysis.
Meaning that when an operand dies, and is a local,
it will automatically be freed so it can be re-used
when a new local is required. The result of this, is
a lower allocation required for locals. Having less
locals means smaller binary size, as well as faster
compilation speed when loaded by the runtime.
* When a field starts at some bit offset within a byte you need to load
starting from that byte and shift, not starting from the next byte,
so a rounded-down divide is required here, not a rounded-up one.
* Remove paragraph from doc that no longer relates to anything.
Closes#12363
When we want a runtime pointer to a zero-bit value we use an undef
pointer, but what if we want a runtime pointer to a comptime-only value?
Normally, if `T` is a comptime-only type such as `*const comptime_int`,
then `*const T` would also be a comptime-only type, so anything
referencing a comptime-only value is usually also comptime-only, and
therefore not emitted to the executable.
However, what if instead we have a `*const anyopaque` pointing to a
comptime-only value? Certainly, `*const anyopaque` is a runtime type,
and so we need some runtime value to store, even when it happens to be
pointing to a comptime-only value. In this case we want to do the same
thing as we do when pointing to a zero-bit value, so we use
`hasRuntimeBits` to handle both cases instead of ignoring comptime.
Closes#12025
Due to the unavailability of fchdir in Windows, a call for setting the
CWD needs to either call chdir with the path string or call
SetCurrentDirectory.
Either way, since we are dealing with a Handle in Windows, a call for
GetFinalPathNameByHandle is necessary for getting the file path first.
This moves functions that LLVM generates calls to,
to the compiler_rt implementation itself, rather than c.zig.
This is a prerequisite for native backends to link with compiler-rt.
This also allows native backends to generate calls to `memcpy` and the like.
The definition of HKEY__ as a struct with an unused int field is only the case in the Windows headers when `STRICT` is defined. From https://learn.microsoft.com/en-us/windows/win32/winprog/enabling-strict:
> When STRICT is defined, data type definitions change as follows:
>
> - Specific handle types are defined to be mutually exclusive; for example, you will not be able to pass an HWND where an HDC type argument is required. Without STRICT, all handles are defined as HANDLE, so the compiler does not prevent you from using one type of handle where another type is expected.
Zig's `opaque {}` already gives this benefit to us, so the usage of a struct with an unused field is unnecessary, and it was causing HKEY to have an alignment of 4, which is a problem because there are HKEY constants like HKEY_LOCAL_MACHINE (0x80000002) that are not 4-byte aligned. Without this change, the compiler would not allow something like HKEY_LOCAL_MACHINE to be defined since it enforces pointer alignment.
This makes the following changes for i386:
long long and unsigned long long have 4 byte alignment on non-Windows
f64 (double) has 4-byte alignment on non-Windows
long double is 80 bits and has 4 byte alignment on mingw
long double on android is 64 bits, not 80: https://www.uclibc.org/docs/psABI-i386.pdfFixes#12453Fixes#12987
This change also exposes some of the existing functions under both the
PPC-style names symbols and the compiler-rt-style names, since Zig
currently lowers softfloat calls to the latter.
Stage 2's softfloat support still had a couple of gaps, which were
preventing us from lowering `f16` on this target. With any luck,
this is enough to get PPC64 working as a Tier 2 target again.
I think this may be helpful in the future when we might want to calculate this again at some other point.
It also makes it more clear that the other two functions below it are only required for this calculation.
* prep: output_buffer -> output_buffer_slice
* fix: truncate lines accurately
Currently, the code assumes a terminal width of 100.
If we look at what's printed for the last test:
```
Test [1/1] test "basic functionality"... [101/100] this is a really long name designed to activate the truncation code. let's fi...
```
No, it does not really work because the relevant part here is `"[101/100] this is a really long name designed to activate the truncation code. let's fi... "`,
which is 90 characters, but we expect 100 because that's the width that is assumed.
The reason is that it also measures **unprintable characters** (escape sequences) at least non-Windows systems.
With this commit the output is now:
```
Test [1/1] test "basic functionality"... [101/100] this is a really long name designed to activate the truncation code. let's find out if...
```
Of which `"[101/100] this is a really long name designed to activate the truncation code. let's find out if... "`
is the actual output of *our* `std.Progress` (remember that `zig test` has an `std.Progress` and our test itself does).
The length of that string is 100. Now the length is consistent with Windows where we don't use escape sequences. This issue was only present on non-Windows systems.
* feat: decide optimal maximum width
This is done by 1. getting the current terminal width and 2. subtracting that by the current cursor column. This accounts for previous output from someone else.
* test: add more tests
They make it easier to see how the progress line is printed in different cases.
* style: fix typo and improve docs
It also expands an acronym used as a variable name. It confused me.
* cleanup: import std.time
* test: add test
* fix: limit termios usage to Linux only for now
* fix: missing cast on Windows
* test: try to debug failure
* fix: fix off-by-one and disable tests
* docs: make comment clearer
* fix: more durability
* fix(getTerminalWidth): change order
Previously, you might obtain `-lLLVM-15` from the CMake configuration,
but we might not be able to locate the library if it's not in your
system library path.
Some architectures (AMDGPU) do not support atomic exchange/fetch for
small types (for AMDGPU: 8- and 16-bit ints). For these types
atomic fetch and atomic exchange needs to be implemeted using atomic
operations on a wider type using cmpxchg.
This commit changes the way Zig is intended to deal with variable
declaration for exotic targets. Where previously the idea was to
enfore local/global variables to be placed into their respective
address spaces, depending on the target, this is now fixed to the
generic address space.
To facilitate this for targets where local variables _must_ be
generated into a specific address space (ex. amdgcn where locals
must be generated into the private address space), the variable
allocations (alloca) are generated into the right address space
and then addrspace-casted back to the generic address space. While this
could be less efficient in theory, LLVM will hopefull deal with figuring
out the actual correct address space for a pointer for us. HIP seems to
do the same thing in this regard.
Global variables are handled in a similar way.
build.zig:
- use "-I" instead of "-isystem" for `b.addSearchPrefix()`
main.zig:
- silently ignore superfluous search dirs
- warn when a dir is added to multiple searchlists
- consolidate "expected paramter after {s}" fatal error messages
- rename command-line switch `-dirafter` → `-idirafter`
closes#12888
Ideally this duplicated code could be factored out into a function, but
there doesn't seem to be any way in the Zig type system to represent an
argument to a function called at comptime that is only needed if it is
comptime-known. Instead, we document what is going on in an adjacent
comment in case it gets copy-pasted into new methods in the future.
When trying to allocate memory for functions like `Managed.init` and
`Managed.set` on the stack, a comptime-known allocation size is desired.
The doc comments for these functions indicate that `calcLimbLen` can be
used to determine how many limbs to allocate, but if `value` is not
comptime-known, then neither is `calcLimbLen(value)`. However, an upper
bound on the allocation size is still computable at comptime in this
case, so this note documents an expression that can be used, rather than
trying to add it to every doc comment that mentions `calcLimbLen`.
This implements `@export(a.b, .{..});` in semantic analysis,
allowing users to directly export a variable from a namespace.
* add test case for exporting using field access
This allows converting a comptime_int to an optional integer type, which
either behaves the same as an implicit cast or produces null if the
argument is outside the range of the destination type.
Without the packed qualifier, the type layout that we use to
initialize doesn't match the correct layout of the underlying
storage, causing corrupted data and past-the-end writes.
When testing the Wasm linker for the producers section
we do not ever want to strip the binary as this will remove
the producers section in release-small.
This fixes the CI errors by d086b371f0
Thanks to Martin Storsjö for explaining this to me on IRC:
__USE_MINGW_ANSI_STDIO redirects stdio functions towards mingw-w64
reimplementations of them (since msvcrt.dll lacks lots of things). For
x86 with "long double", this is also needed to get long doubles
formatted properly. It's enabled by default by headers when building in
C99 mode, unless you're targeting UCRT. The headers normally enable this
automatically - or you can request it enabled with
-D__USE_MINGW_ANSI_STDIO=1. However, the mingw-w64-crt files are
expected to be built with this explicitly turned off. Since there's a
half dozen various ways of configuring the CRT and various features, the
mingw-w64-crt files specifically need to be built in a very hardcoded
configuration, which is different from how end user source files are
compiled.
This commit removes a patch that we were carrying previously.
See #7356
These tests will be failing on many platforms until #8465 is resolved.
Luckily, the particular function signature used for __divXc3 and __mulXc3
seems to be OK on x86-64.
These are the standard complex multiplication/division functions
required by the C standard (Annex G).
Don't get me started on the standard's handling of complex-infinity...
This re-write was needed to fix deficiencies in the existing ldexp,
which was failing to compute correct results for both f16 and f80.
It would be nice to add a fast multiplication-based fallback in the
future for targets that have a hardware FPU, but this implementation
should be much faster than the existing for targets without one.
Previously, you might obtain `-lLLVM-15` from the CMake configuration,
but we might not be able to locate the library if it's not in your
system library path.
Addends in relocations are signed integers as theoretically it could
be a negative number. As Atom's offsets are relative to their parent
section, the relocation value should still result in a postive number.
For this reason, the final result is stored as an unsigned integer.
Also, rather than using `null` for relocations that do not support
addends. We set the value to 0 for those that do not support addends,
and have to call `addendIsPresent` to determine if an addend exists
or not. This means each Relocation costs 4 bytes less than before,
saving memory while linking.
This also turns off non-debug modes for the bss linker tests for
Wasm. This is done as it's not required to guarantee to zero out
the bss section for non-debug modes.
The `producers` section contains meta data of the binary and/or
object file. It *can* contain the source language, the tool it
was processed by, and/or the SDK that was used to produce the file.
For now, we always set the language and processed-by fields to Zig.
In the future we will parse linked object files to detect their
producers sections and append (if different) their language, SDK
and processed-by fields.
Windows requires the directory handle to be closed before attempting to delete the directory, so now we do that and then re-open it if we need to retry (from getting DirNotEmpty when trying to delete).
This was sized large so that `getdents` (and other platforms' equivalents) could provide large amounts of entries per syscall, but some benchmarking seems to indicate that the larger 8192 sizing doesn't actually lead to performance gains outside of edge cases like extremely large amounts of entries within a single directory (e.g. 25,000 files in one directory), and even then the gains are minimal ('./walk-8192 dir-with-tons-of-entries' ran 1.02 ± 0.34 times faster than './walk-1024 dir-with-tons-of-entries').
Note: Sizes 1024 and 2048 had similar performance characteristics, so the smaller of the two was chosen.
`deleteTree` now uses a stack-allocated stack for the first 16 nested directories, and then falls back to the previous implementation (which only keeps 1 directory open at a time) when it runs out of room in its stack. This allows the function to perform as well as a recursive implementation for most use-cases without needing allocation or introducing the possibility of stack overflow.
There are two parts to this:
1. The deleteFile call on the sub_path has been moved outside the loop, since if the first call fails with `IsDir` then it's very likely that all the subsequent calls will do the same. Instead, if the `openIterableDir` call ever hits `NotDir` after the `deleteFile` hit `IsDir`, then we assume that the tree was deleted at some point and can consider the deleteTree a success.
2. Inside the `dir_it.next()` loop, we look at entry.kind and only try doing the relevant (deleteFile/openIterableDir) operation, but always fall back to the other if we get the relevant error (NotDir/IsDir).
Before this commit:
```
$ zig test lib/std/fs/test.zig --main-pkg-path lib/std --zig-lib-dir lib
2170 passed; 37 skipped; 0 failed.
```
After this commit:
```
$ zig test lib/std/fs/test.zig --main-pkg-path lib/std --zig-lib-dir lib
All 45 tests passed.
```
This matches stage1 behavior:
```
$ zig test -fstage1 lib/std/fs/test.zig --main-pkg-path lib/std --zig-lib-dir lib
All 45 tests passed.
```
All tests are still run if `zig test` is run directly on `lib/std/std.zig`:
```
$ zig test lib/std/std.zig --main-pkg-path lib/std --zig-lib-dir lib
2170 passed; 37 skipped; 0 failed.
```
`zig build test-std` is unaffected by this change.
Closes#12926
This was an accidental misuse of the Cache API which intends to call
resolve on all file paths going into it. This one callsite was failing
to do that; fixed now.
Fixes relative file paths from making it into the global cache manifest.
See #13050
This implements the new addition to the API: `sock_accept`.
Reference commit of WASI spec:
0ba0c5e2e37625ca5a6d3e4255a998dfaa3efc52
For full details:
0ba0c5e2e3
For entire spec at this commit:
0ba0c5e2e3/phases/snapshot/docs.md
expected type 'fn() void', found 'fn(i32) void'
function with 0 parameters cannot cast into a function with 0 parameters
=>
expected type 'fn() void', found 'fn(i32) void'
function with 1 parameters cannot cast into a function with 0 parameters
Superceeds PR #12735 (now supporting all packed structs in GNU C)
Fixes issue #12733
This stops translating C packed struct as a Zig packed struct.
Instead use a regular `extern struct` with `align(1)`.
This is because (as @Vexu explained) Zig packed structs are really just integers (not structs).
Alignment issue is more complicated. I think @ifreund was the
first to notice it in his comment on PR #12735
Justification of my interpretion of the C(lang) behavior
comes from a careful reading of the GCC docs for type & variable attributes:
(clang emulates gnu's packed attribute here)
The final line of the documentation for __attribute__ ((aligned)) [on types] says:
> When used on a struct, or struct member, *the aligned attribute can only increase the alignment*; in order to decrease it, the packed attribute must be specified as well.
This implies that GCC uses the `packed` attribute for alignment purposes
in addition to eliminating padding.
The documentation for __attribute__((packed)) [on types], states:
> This attribute, attached to a struct, union, or C++ class type definition, specifies that each of its members (other than zero-width bit-fields) is placed to minimize the memory required. **This is equivalent to specifying the packed attribute on each of the members**.
The key is resolving this indirection, and looking at the documentation
for __attribute__((packed)) [on fields (wierdly under "variables" section)]:
> The packed attribute specifies that a **structure member should have the smallest possible alignment** — one bit for a bit-field and one byte otherwise, unless a larger value is specified with the aligned attribute. The attribute does not apply to non-member objects.
Furthermore, alignment is the only effect of the packed attribute mentioned in the GCC docs (for "common" architecture).
Based on this, it seems safe to completely substitute C 'packed' with Zig 'align(1)'.
Target-specific or undocumented behavior potentially changes this.
Unfortunately, the current implementation of `translate-c` translates as
`packed struct` without alignment info.
Because Zig packed structs are really integers (as mentioned above),
they are the wrong interpretation and we should be using 'extern struct'.
Running `translate-c` on the following code:
```c
struct foo {
char a;
int b;
} __attribute__((packed));
struct bar {
char a;
int b;
short c;
__attribute__((aligned(8))) long d;
} __attribute__((packed));
```
Previously used a 'packed struct' (which was not FFI-safe on stage1).
After applying this change, the translated structures have align(1)
explicitly applied to all of their fields AS EXPECTED (unless explicitly overriden).
This makes Zig behavior for `tranlsate-c` consistent with clang/GCC.
Here is the newly produced (correct) output for the above example:
```zig
pub const struct_foo = extern struct {
a: u8 align(1),
b: c_int align(1),
};
pub const struct_bar = extern struct {
a: u8 align(1),
b: c_int align(1),
c: c_short align(1),
d: c_long align(8),
};
```
Also note for reference: Since the last stable release (0.9.1),
there was a change in the language spec
related to the alignment of packed structures.
The docs for Zig 0.9.1 read:
> Packed structs have 1-byte alignment.
So the old behavior of translate-c (not specifying any alignment) was possibly correct back then.
However the current docs read:
> Packed structs have the same alignment as their backing integer
Suggsestive both to the change to an integer-backed representation
which is incompatible with C's notation.
When encountering a fn type that returns an error (union), a backend
that supports error return tracing will want the StackTrace struct and
its fields to be analyzed.
Previously AstGen would set decl_line for containers so that
declarations inside them would be relative to the start of the
container but Sema was not aware of the line offset of the container
and would make them relative to the containers parent decl which
would then break for generic structs.
In the future when working on incremental compilation it will likely
be better to communicate the line delta to Sema but for now this
is a simpler fix that correctly handles the non-incremental case.
Closes#12725Closes#12818
* the root struct decl name is fully qualified
this prevents error messages containing 'main.main'
* avoid declared here note when file struct is missing a member
It always points at the start of the file which might contain another
container misleading the user.
The container we want to get the fields from might not be declared in the
same file as the block we are analyzing, so we should get the AST from
the decl's file instead.
The enum we want to get the fields from might not be declared in the
same file as the block we are analyzing, so we should get the AST from
the decl's file instead.
Closes#12950.
These const qualifiers on pointers to opaque types do not serve any
purpose. If anything they are misleading since the underlying pointers
very likely point to objects that are in fact mutated.
This commit does not change any behavior.
cmpxchg_weak and cmpxchg_strong are not very common; demote them to
extended operations to make some headroom.
This commit does not change any behavior, only memory layout of the
compiler.
If `-o` is not specified for `zig cc` for linking, it should emit the
`a.out` (or equivalent) to the current directory. So then this:
$ zig cc test.c
Should result in `./a.out`.
If directory is omitted from `Compilation.EmitLoc`, it will default to
the cache directory later in the compilation chain, which will result
in `a.out` missing in the current directory.
If we specify the directory to `Compilation.EmitLoc`, it will be
handled, allegedly, correctly.
Fixes#12858
Previously we were collecting as autodoc decls everything that was a ZIR
decl in a rather naive way. Now we only collect decltests as part of the
data relative to the decl they refer to, and ignore everything else.
For example, building stage2 requires an empty `empty.cc` source file
compiling which generates a valid translation unit with no symtab/strtab.
In this case, we cannot simply assume that every translation unit will
have a valid symtab; instead, we cautiously default the input symtab
and strtab fields to optional `null` to signal symtab's presence or its lack of.
In case the symtab is not present, we catch this fact when splitting
input sections into subsections and create a synthetic symbol per every
suitable section.
Many of these tests check for the incorrect behavior of stage1 whereas
self-hosted correctly does not emit an error, so they are simply
deleted.
The remaining number of test cases within the stage1/ subdirectory is
reduced from 143 to 103.
No longer introduce build options for tests. Instead, ZIG_EXE
environment variable is added to any invocation of `zig run` or `zig
test`.
The end result of this branch is the same: there is no longer a
mandatory positional command line argument when invoking zig test
binaries directly.
This commit makes `zig cc` match the equivalent behavior of
`zig build-exe` with regards to caching. That is - it will cache
individual .c to .o compilations (with the usual exceptions), but will
always repeat the linking process so that incremental linking has a
chance to happen.
Perhaps a future enhancement will provide a way to get the old behavior,
but I suspect this new behavior will be preferred by everyone, because
it is closer to what C compilers do. Note that the old behavior can be
obtained by switching to `zig build-exe` instead of `zig cc` and using
the `--enable-cache` parameter.
Closes#12317
This reverts commit d31be31267.
The problem was happening due to an LLVM bug exposed by having LTO
enabled for libunwind. The simple workaround is to disable LTO for
libunwind. It can be re-enabled in the future when the upstream bug
is fixed.
See #12828
This bug manifested as a segfault in stage1 when calling this function.
The C++ code looks like this:
```c++
entry->llvm_di_type = ZigLLVMCreateDebugForwardDeclType(g->dbuilder,
ZigLLVMTag_DW_structure_type(), full_name,
import ? ZigLLVMFileToScope(import->data.structure.root_struct->di_file) : nullptr,
import ? import->data.structure.root_struct->di_file : nullptr,
line);
```
There is actually no problem here - what happened is that because
cross-language LTO was enabled between zig and c++ code, and because
Zig annotated the file parameter (3rd line) as being non-null, the C++
code assumed that parameter could not be null, and eagerly dereferenced
`import->...`, causing a segfault, since it was null.
I verified that this commit fixed the problem and I also verified this
hypothesis by disabling LTO and noticing that it indeed avoided the
problem.
This makes translate-c lower discards as `_ = @TypeOf(foo);` to avoid
tripping the "pointless discard" error.
Ideally, translate-c would avoid emitting pointless discards, in which
case this commit can be reverted, however, that is a separate
enhancement.
The specification for this function is that it returns a positive value,
zero, or negative value, not that it returns the difference between
ascii values.
I'm not sure why the other commits in this branch caused this fix to be
necessary. Also, there seems to be more fixes necessary before tests
will pass.
The changes from https://reviews.llvm.org/D119173 mean that __config no
longer defaults the libc++ ABI to 1, relying on external configuration.
This means Zig must provide the external configuration.
This fixes static libraries built with zig with -lc++ to have the
standard __1 namespace prefix, which had previously regressed in the
llvm15 branch.
On each invocation of `flush()` the file pointer is moved.
This means that rather than overwriting the binary file,
we're appending to the file. With this commit, we're resetting
said pointer to '0' and overwrite the existing binary in incremental
mode.
Rather than writing to the file using a writer, we now first write to
an arraylist and store the binary in memory. Once the full binary
data was written, we write all data to disk at once. This reduces
the amount of syscalls tremendously, increasing the performance of
the linker in exchange for increased memory usage during flush.
By writing them at the very end, we can easily detect
where the writing of the binary went wrong as tools will
indicate the missing of the magic bytes.
Storing defers this way has the benefits that the defer doesn't get
analyzed multiple times in AstGen, it takes up less space, and it
makes Sema aware of defers allowing for 'unreachable else prong'
error on error sets in generic code.
The disadvantage is that it is a bit more complex and errdefers with
payloads now emit a placeholder instruction (but those are rare).
Sema.zig before:
Total ZIR bytes: 3.7794370651245117MiB
Instructions: 238996 (2.051319122314453MiB)
String Table Bytes: 89.2802734375KiB
Extra Data Items: 430144 (1.640869140625MiB)
Sema.zig after:
Total ZIR bytes: 3.3344192504882812MiB
Instructions: 211829 (1.8181428909301758MiB)
String Table Bytes: 89.2802734375KiB
Extra Data Items: 374611 (1.4290275573730469MiB)
When linking libc and compiling natively, Zig tries to integrate with
the system C compiler. However, this caused Zig to fail when no system C
compiler is installed, despite the fact that Zig is perfectly capable of
compiling & linking libc without one.
This commit makes Zig fall back to using its own ability to provide libc
in the case that no C compiler is installed. For glibc, it means
sometimes getting the warning "zig cannot build new glibc version abc,
providing instead xyz".
Ideally, Zig would do some more validation about the system libraries
being linked against, and report an error in case it could not provide
the exact correct libc version of the system libraries (or that the
system libraries themselves conflict with each other), however, I think
it is fair to call that a separate enhancement.
Before, Zig tried to use its own libc files (e.g. glibc) when there were
no system libs being linked. This prevented building against native
glibc on systems that have newer glibc than the ones Zig provides.
Closes#12797
this commit removes whitespace and changes Decl, AstNode and Type to be
json arrays instead of json objects. This change reduces json payload
size for the stdlib from 25mb to < 10mb.
Macro definitions are simply a slice of bytes, which may not be
UTF-8 encoded. If they are not UTF-8 encoded, escape non-printable
and non-ASCII characters as `\xNN`.
Fixes#12784
- add missing checks whether destination fits into the operand
- remove reuseOperand invocations from airIsNullPtr and similar
functions as we need to load the operands into temporary locations
After failing to find RUNPATH in the ELF of /usr/bin/env, not finding
the answer in a symlink of the dynamic interpreter, and not finding
libc.so.6 in the same directory as the dynamic interpreter, Zig will
check `/lib/$triple`.
This fixes incorrect native glibc version detected on Debian bookworm.
This is a partial revert of the previous commit, fixing a regression on
Debian. However, the commit additionally improves the
detectAbiAndDynamicLinker function to read more than 1 byte at a time
when detecting a shebang line.
This commit removes the check that takes advantage of when the dynamic
linker is a symlink. Instead, it falls back on the same directory as the
dynamic linker as a de facto runpath. Empirically, this gives correct
results on Gentoo and NixOS.
Unfortunately it is still falling short for Debian, which has libc.so.6
in a different directory as the dynamic linker.
Before, native glibc and dynamic linker detection attempted to use the
executable's own binary if it was dynamically linked to answer both the
C ABI question and the dynamic linker question. However, this could be
problematic on a system that uses a RUNPATH for the compiler binary,
locking it to an older glibc version, while system binaries such as
/usr/bin/env use a newer glibc version. The problem is that libc.so.6
glibc version will match that of the system while the dynamic linker
will match that of the compiler binary. Executables with these versions
mismatching will fail to run.
Therefore, this commit changes the logic to be the same regardless of
whether the compiler binary is dynamically or statically linked. It
inspects `/usr/bin/env` as an ELF file to find the answer to these
questions, or if there is a shebang line, then it chases the referenced
file recursively. If that does not provide the answer, then the function
falls back to defaults.
This commit also solves a TODO to remove an Allocator parameter to the
detect() function.
Previously, this code would fail to detect glibc version because it
relied on libc.so.6 being a symlink which revealed the answer. On modern
distros, this is no longer the case.
This new strategy finds the path to libc.so.6 from /usr/bin/env, then
inspects the .dynstr section of libc.so.6, looking for symbols that
start with "GLIBC_2.". It then parses those as semantic versions and
takes the maximum value as the system-native glibc version.
closes#6469
see #11137closes#12567
This adds the following for passthrough to lld:
- `--print-gc-sections`
- `--print-icf-sections`
- `--print-map`
I am not adding these to the cache manifest, since it does not change
the produced artifacts.
Tested with an example from #11398: it successfully prints the resulting
map and the GC'd sections.
If there are zerofill sections, the loader may copy the contents of
the physical space in file directly into memory and attach that to
the zerofill section. This is a performance optimisation in the loader
but requires us, the linker, to properly zero-out any space between
__DATA and __LINKEDIT segments in file. This is of course completely
skipped if there are no zerofill sections present.
This is a temporary workaround to an unclear platform-dependence
behavior we have in libstd for `std.fs.File` abstraction. See
https://github.com/ziglang/zig/issues/12783 for more information.
Previously if a decl failed its capture scope would be deallocated and
set to undefined which would then lead to invalid dereference in
`zirClosureGet`. To avoid this set the capture scope to a special
failed state and fail the current decl with dependency failure if
the failed state is encountered in `zirClosureGet`.
Closes#12433Closes#12530Closes#12593
This relieves register pressure, and reduce generated code size
(since now we can use the same index register for both `mov_scale_src`
and `mov_scale_dst` MIR instructions).
Fix lowering of ModRM + SIB encodings where index register is extended
- previously, we would carelessly ignore the fact generating incorrect
encodings.
As far as I can see, unlike with MachO, we don't have any stubs
helper routines available and need to load a bound pointer into
a register to then call it.
Although the wasm-linker previously already supported
debug information in incremental-mode, this was no longer
working as-is with the addition of supporting object-file-parsed
debug information. This commit implements the Zig-created debug information
structure from scratch which is a lot more robust and also allows
being linked with debug information from other object files.
When linking a Zig-compilation with an object file,
we allow mixing the debug atoms to make sure debug
information is preserved from object files. By default,
we now always initialize all debug sections if the `strip` flag
is unset.
This also fixes relocations for debug information as previously
the offset of an atom wasn't calculated, and neither was the code
size itself which meant that debug lines were off and file names
from other object files were missing.
Previously we used single arraylists for each debug section for debug
information that was generated from Zig code. (e.i. `Module` is available).
This information is now stored in Atoms, similarly to debug information
from object files. This will allow us to link them together and resolve
debug relocations.
This correctly performs a relocation for debug sections.
The result is that the wasm-linker can now correctly create
a binary from object files while preserving all debug information.
We now link relocatable debug sections with the correct
section symbol and then allocate and resolve the debug atoms
before writing them into the final binary.
Although this does perform the relocation, the actual relocations
are not done correctly yet.
Rather than storing the name of a debug section into the structure
`RelocatableData`, we use the `index` field as an offset into the
debug names table. This means we do not have to store an extra 16 bytes
for non-debug sections which can be massive for object files where each
data symbol has its own data section. The name of a debug section
can then be retrieved again when needed by using the offset and
then reading until the 0-delimiter.
Currently, `zig build-exe -fno-emit-bin --verbose-air src/main.zig`
results in no output at all. With this refactor, it dumps AIR
and then exits without invoking LLVM, as expected
saying []T is a pointer is confusing because zig docs say there are two types of pointers (*T and [*]T). It is more clear to say that []T is a slice type which contains a [*]T pointer and a length.
Co-authored-by: Philipp Lühmann <47984692+luehmann@users.noreply.github.com>
The original impetus for making a change here was a typo in --add-header
causing the script to fail. However, upon inspection, I was alarmed that
we were making a --recursive upload to the *root directory* of
ziglang.org. This could result in garbage files being uploaded to the
website, or important files being overwritten. As I addressed this concern,
I decided to take on file compression as well.
Removed compression prior to sending to S3. I am vetoing pre-compressing
objects for the following reasons:
* It prevents clients from working which do not support gzip encoding.
* It breaks a premise that objects on S3 are stored 1-to-1 with what is
on disk.
* It prevents Cloudflare from using a more efficient encoding, such as
brotli, which they have started doing recently.
These systems such as Cloudflare or Fastly already do compression on
the fly, and we should interop with these systems instead of fighting them.
Cloudfront has an arbitrary limit of 9.5 MiB for auto-compression. I looked
and did not see a way to increase this limit. The data.js file is currently
16 MiB. In order to fix this problem, we need to do one of the following things:
* Reduce the size of data.js to less than 9.5 MiB.
* Figure out how to adjust the Cloudfront settings to increase the max size
for auto-compressed objects.
* Migrate to Fastly. Fastly appears to not have this limitation. Note
that we already plan to migrate to Fastly for the website.
This means we can request ASLR on by default as other COFF linkers
do. Currently, we write the base relocations in bulk, however,
given that there is a mechanism for padding in place in PE/COFF
I believe there might be room for making it an incremental operation
(write base relocation whenever we add/update a pointer that would
require it).
Previously, this function used incorrect registers for the munmap syscall, leading to detached threads not cleaning up.
closes#12690
Co-authored-by: bxlr <biexelar@diroot.org>
When instantiating a generic function, there is a period of time where
the function is inserted into monomorphed_funcs map, but is not yet
initialized. Despite semantic analysis being single-threaded, generic
function instantiation can happen recursively, meaning that the hash
and equality functions for monomorphed_funcs entries are potentially
invoked for an uninitialized function.
This problem was mitigated by pre-setting the hash field on the newly
allocated function, however it did not solve the problem for hash
collisions in which case the equality function would be invoked. That it
was solved for hash() but not eql() explains why the problem was
difficult to observe. I tested this patch by temporarily sabotaging the
hash and making it always return 0.
This fix is centered on adding a new field to Module.Fn which is the one
checked by eql() and is populated pre-initialization.
closes#12643
Extern functions were missing attributes such as "readonly" on
non-optional pointers, and "byval" which is required to match C ABI.
Follow-up from bf28765a97.
closes#12683
This is problematic because in practice it depends on whether the
compiler backend supports it too, as evidenced by the TODO comment about
LLVM not supporting some architectures that in fact do support tail
calls.
Instead this logic is organized strategically in src/target.zig, part of
the internal compiler source code, and the behavior tests in question
duplicate some logic for deciding whether to proceed with the test.
The proper place to expose this flag is in `@import("builtin")` - the
generated source file - so that third party compilers can advertise
whether they support tail calls.
This also fixes performing relocations for data symbols
of which the target symbol exists in an external object file.
We do this by checking if the target symbol was discarded,
and if so: get the new location so that we can find the
corresponding atom that belongs to said new location. Previously
it would always assume the symbol would live in the same file
as the atom/symbol that is doing the relocation.
Generate symbols for extern variables and try to resolve them.
Unresolved 'data' symbols generate an error as they cannot be
exported from the Wasm runtime into a Wasm module. This means,
they can only be resolved by other object files such as from other
Zig or C code compiled to Wasm.
Given that COFF will want to support PIC from ground-up, there is no
point in leaving outdated code for COFF in other backends such as
arm or aarch64. Instead, when we are ready to look into those, we
can start figuring out what to add and where.
This is not technically correct, but given that we are not yet able
to link against the CRT, it's a good default until then.
Add basic logging of generated symbol table in the linker.
Regardless of the build mode (build-exe, build-lib), always
set the default stack size to 1MB. Previously, this was only
done when using build-exe, making the inconsistancy confusing.
The user can still override this behavior by providing the
`--stack <size>` flag.
* CMakeLists: pass `-Dstrip` for release zig builds
* pass -target and -mcpu to zig1. works around llvm on freebsd
incorrectly detecting "freestanding" instead of "freebsd" for the
native OS.
* ci.ziglang.org is now responsible for creating aarch64-macos tarballs
rather than Azure.
Adds a `unused: u32 = 0` field to `Zir.Header`.
We could leave this as padding, however it triggers a Valgrind warning because
we read and write undefined bytes to the file system. This is harmless, but
it's essentially free to have a zero field here and makes the warning go away,
making it more likely that following Valgrind warnings will be taken seriously.
This is a simplification of the cmake build script which introduces a
new "stage3" target that is built by default, which builds and installs
a stage3 zig.
It greatly simplifies the build instructions for Zig, making it conform
to the regular cmake routine, while still producing a stage3 artifact.
Since now the size of a c_longdouble is correctly 16 bytes,
the test is no longer passing. It was previously accidentally passing
due to incorrect sizing and it not being larger than the size
of a f64.
disable long_double test for windows
Adds error for taking a non comptime parameter in a function returning a
comptime-only type but not when that type is dependent on a parameter.
Co-authored-by: Veikka Tuominen <git@vexu.eu>
According to https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md
the size of c's long double is 16 bytes for Wasm, rather than 8 bytes
which was the value previously in the compiler. This ensures
we not only pass the correct value, but also creates the correct
function signature needed to pass the Wasm validator.
This also adds an additional test case in c_abi tests.
`sc_fpstate` member of `struct sigcontext` is a `struct fxsave64 *`.
use *anyopaque to represent it.
avoid to defining `fxsave64` as it is a packed struct with some arrays.
From `copy_file_range(2)` errors:
ETXTBSY
Either fd_in or fd_out refers to an active swap file.
Same error will be used in the upcoming `ioctl_ficlonerange(2)`:
ETXTBSY
One of the files is a swap file. Swap files cannot share storage.
We call `sema.resolveTypeFields` in order to get the fields of structs
and unions inserted into their data structures. If it isn't called, it
can happen that the fields of a type is queried before those fields are
inserted into (for instance) `Module.Union.fields`, which would result in
a wrong 'no field named' error.
Fixes: #12486
A self-defined macro is one of the form `#define FOO FOO`
Those types of macros have never been translated; this change will cause
any macros which refer to them to be translated as `@compileError` instead
of referring to a non-existent identifier.
Closes#12471
Previously, when lowering AIR instructions `wrap_errunion_payload`,
`wrap_errunion_err`, and `wrap_optional`, the LLVM backend would create
an alloca instruction to store the result, but did not set the alignment
on it. This caused UB which went undetected for a long time until we
started enabling the stack protector.
Closes#12594
Unblocks #12508
Inspires #12634
Tests passed locally:
* test-behavior
* test-cases
Empirically, the ReleaseSmall std lib tsets take about 55 minutes on the
CI, and is the bottleneck causing timeouts. So this commit disables full
coverage in favor of running a smaller set of ReleaseSmall std lib tests.
This commit enables `-u <symbol>` for ELF and `-include:<symbol>` for
COFF linkers for use internally. This means we do not expose these
flags to the users just yet, however, we make use of them internally
whenever required. One such use case is forcing inclusion of
`_tls_index` when linking for Windows with mingw and LTO and dead
code stripping enabled. This ensures we add `_tls_index` to the symbol
resolver as an undefined symbol and force the linker to include an atom
that provides it marking it a dead-code-stripping root - meaning it will
not be garbage collected by the linker no matter what.
This fixes a bug exposed by cd1833044a
where a function type would be converted to generic_poison even after
being instantiated due to containing comptime only types.
This could also be fixed by just checking `is_generic_instantiation`
but this way also provides better type names.
Closes#12625
Commit eb3f7d2f37 changed the order of CLI
args passed to clang, making object-specific "extra flags" passed first.
However, these are supposed to be able to override other flags, and this
behavior is exploited by workarounds in mingw.zig to disable LTO.
This commit rectifies the situation by moving extra flags back to being
passed after the call to addCCArgs().
Previously, Zig had inconsistent semantics for an enum like this:
`enum(u8){zero = 0}`
Although in theory this can only hold one possible value, the tag
`zero`, Zig no longer will treat the type this way. It will do loads and
stores, as if the type has runtime bits.
Closes#12619
Tests passed locally:
* test-behavior
* test-cases
When removing generic function instantiations from monomorphed_funcs, we
need to first make sure the function is generic, otherwise the hash map
tries to access the `hash` field of the function which is undefined.
closes#12614
When the entire purpose of this compilation is to perform a single zig
cc operation we could "tail call" clang by doing an execve, and any use
of the caching system would actually be problematic since the user is
presumably doing their own caching by using dep file flags.
Fixes#12317
This reverts commit 0f01e812ff.
This does not belong in `std.posix`, and it is already provided at
`std.os.windows.ws2_32.INVALID_SOCKET`.
See related issue #5019.
This reverts 6d679eb2bc and additionally
changes the command line parameters passed to Clang to match.
Clang 14 defaults to DWARFv5 which is an interesting choice. v5 has been
out for 5 years and yet Valgrind does not support it, and apparently
neither does either GDB or LLD, I haven't determined which, but I wasn't
able to use GDB to debug my LLVM-emitted dwarf 5 zig code that was linked
with LLD.
A couple years ago when I was working on the self-hosted ELF linker, I
emitted DWARFv5 but then downgraded to v4 when I realized that third
party tools were stuck in the past. Years later, they still are.
Hopefully, Clang 14's bold move will inspire third party tools to get
their shit together, however, in the meantime, everything's broken, so
we're passing `-gdwarf-4` to clang and instructing LLVM to emit DWARFv4.
Note that Zig's std.debug code *does* support DWARFv5 already as of a
previous commit that I made today.
We now do not allocate memory for headers and other metadata unless
requested by the caller. Instead, we read-in the entire contents
of the image into memory and operate on pointers and casts wherever
possible. I have a left a TODO to hook up Windows' memory-mapped API
here in-place of standard `readToEndAlloc` which should be more memory
proof on memory constrained hosts.
This commit also supplements our `std.coff` with a lot missing basic
extern structs required to make our COFF linker.
This `pdb.Pdb.init` call can return `error.FileNotFound`, which was previously resulting in:
Unable to print stack trace: FileNotFound
which also aborts the stack trace printing (so any deeper stack traces are not printed).
It makes more sense to treat it as `MissingDebugInfo` which then gets printed as:
???:?:?: 0x7fffa8817033 in ??? (???)
and allows the stack trace to continue printing.
Note: locally, the error.FileNotFound was being triggered for me when looking for kernel32.pdb and ntdll.pdb
* Added support for stroffsetsptr class in Dwarf stdlib
* Proper initializion of debug_str_offsets in DwarfInfo
* Added missing null initializer to DwarfInfo in Macho
* Added missing is_64 field to getAttrString in DwarfInfo
* Fixed formatting
* Added missing is_64 param to getAttrString
* Added required cast to usize
* Adding missing .debug_str_offsets initialization
* getAttrString now uses the str_offsets_base attr
This reverts commit 1a32f2a7f4.
Sorry, this workaround is not welcome. Instead, please solve the actual
issue by doing the accepted behavior in the compiler itself:
> in a catch or else (handling a returned error), if the block does not
> try or return error.xyz, set the index to 0
This also applies to if statements, such as the one that test runner is
doing just above this hack.
Essentially #1923 means "caught" errors still show up in error return traces.
The correct fix would require the compiler to fix this, but that could affect performance.
For now, simply workaround this issue by clearing the return traces
between tests.
This means that "caught" errors in one test will not show up in the
error traces of other tests.
When an object file is being parsed from within an archive
file, we provide the object file size to ensure we do not
read past the object file. This is because follow up object
files can exist there, as well as an LF character to notate
the end of the file was reached. Such a character is invalid
within the object file.
This also fixes a bug in getting the function/global type
for defined globals/functions from object files as it was missing
the substraction with the import count of the respective type.
Wasm archive files are encoded the same way as GNU.
This means that the header notates the character index within
the long file name list rather than the length of the name.
The entire name is then delimited by an LF character (0x0a).
This also makes a cosmetic update to remove the `self` name,
and rather label it as `archive` instead.
* riscv64: adjust alignment and size of 128-bit integers.
* take ofmt=c into account for ABI alignment of 128-bit integers and
structs.
* Type: make packed struct support intInfo
* fix f80 alignment for i386-windows-msvc
Firstly, opening a file handle is not really needed since we won't even
use it, and secondly, this can cause AccessDenied errors on Windows
when trying to move a directory from zig-cache/tmp/ to zig-cache/o/
since, without POSIX semantics, it is illegal to move directories
with open handles to any of its resources.
This makes `0123` and `u0123` etc. illegal.
I'm now confident that this is a good change because
I actually caught two C header translation mistakes in `haiku.zig` with this.
Clearly, `0123` being octal in C (TIL) can cause confusion, and we make this easier to read by
requiring `0o` as the prefix and now also disallowing leading zeroes in integers.
For consistency and because it looks weird, we disallow it for integer types too (e.g. `u0123`).
Fixes#11963Fixes#12417
This adds additional checks during symbol resolution:
- Ensures function signatures match when a symbol will be replaced.
- Ensures global types match when the symbol is being replaced.
- When both symbols are undefined, ensures they have a matching module name.
Those changes ensure the result will pass the validator when
the runtime compiles the Wasm module.
Additionally, this also slightly changes the behavior when both
the existing symbol and new symbol are both defined. Rather than
always resulting in a collision, it only results in a collision
when both are also weak. Else, the non-weak symbol will be picked.
Rather than storing it in a local and returning that,
we now keep this on the stack as all internal functions
expect it to be on the stack already and therefore were
generating extra `local.set` instructions.
files. Files generated from the standard library could be considered
for placing with main.js and index.html in lib/docs. Paths should
reflect packages in the future.
Move common tests by target file format (Wasm, MachO) into helper
functions in `link.zig`, and sort alphabetically within for easier
tracking versus file organization on disk.
The following, from the documentation as of the time of writing, illustrates
the problem:
```zig
// Compile time coercion of float to int
test "implicit cast to comptime_int" {
var f: f32 = 54.0 / 5;
_ = f;
}
```
It is not clear how to unify the types of 54.0 and 5 to perform the
division. We can either
- cast 54.0 to comptime_int resulting in @as(comptime_int, 10), which is
casted to @as(f32, 10), or
- cast 5 to comptime_float resulting in @as(comptime_float, 10.8), which
is casted to @as(f32, 10.8)
Since the two resulting values are different, a compiler error is appropriate.
If we know that casting to either type will result in the same value we
don't need to error. For instance, 10.0 / 2 is okay, as is 10 / 2.0.
Fixes: #12364
Make sure `ProcSym` includes a single element byte-array which delimits
the start of the symbol's name as part of its definition. This makes
the code more elegant in that accessing the name is equivalent to taking
the address of this one element array.
The `Value.eql` function has to test for value equality *as-if* the lhs
value parameter is coerced into the type of the rhs. For tagged unions,
there was a problematic case when the lhs was an anonymous struct,
because in such case the value is empty_struct_value and the type
contains all the value information. But the only type available in the
function was the rhs type.
So the fix involved making `Value.eqlAdvanced` also accept the lhs type,
and then enhancing the logic to handle the case of the `.anon_struct` tag.
closes#12418
Tests run locally:
* test-behavior
* test-cases
Previously we would assign the error message to Sema and then never
clear it even when destroying the error message, which caused memory
corruption.
Closes#12437
Before this commit, the modified test would fail with `FileNotFound` because the `entry.dir` would be for the entry itself rather than the containing dir of the entry. That is, if you were walking a tree of `a/b`, then (previously) the entry for `b` would incorrectly have an `entry.dir` for `b` rather than `a`.
`isAlNum` and `isAlpha`:
1. I think these names are a bit cryptic.
2. `isAlpha` is a bit ambiguous: is it alpha*numeric* or alpha*betic*?
This is why I renamed `isAlpha` to `isAlphabetic`.
3. For consistency and because `isAlNum` looks weird, I renamed it to `isAlphanumeric`.
`isCntrl`:
1. It's cryptic and hard to find when you look for it.
2. We don't save a lot of space writing it this way.
3. It's closer to the name of the `control_code` struct.
`isSpace`:
1. The name is ambiguous and misleading.
`spaces`:
1. Ditto
`isXDigit`:
1. The name is extremely cryptic.
2. The function is very hard to find by its name.
I think `isBlank` and `isWhitespace` are quite confusable.
What `isBlank` does is so simple that you can just do the `c == ' ' or c == '\t'`
check yourself but in a lot of cases you don't even want that.
`std.ascii` can't really know what you think "blank" means.
That's why I think it's better to remove it.
And again, it seems ambiguous considering that we have `isWhitespace`.
Next, it also deprecates `isGraph`.
It's the same as `isPrint(c) and c != ' '`, which I find confusing.
When something is printable, you can say it also has a *graph*ical representation.
Removing `isGraph` solves this possible confusion.
This makes it so that you can no longer interact with the search bar
or the results or anything while the modal is open. That's why it's a "modal" and not a "dialog".
It also makes it so that you can now always press Esc to return to the results or the main page.
Previously this was only possible when the search field was active.
This indents the keyboard shortcuts related to the S key because you can only
use these keyboard shortcuts after you pressed S (when the search field is focused).
This is a visual hint.
Add handling for these additional `MCValue`s:
* `.immediate` - lower to `DW.OP.consts` or `DW.OP.constu` depending
on signedness followed by popping off the DWARF stack with
`DW.OP.stack_value`
* `.undef` - lower to `DW.OP.implicit_value`
* `.none` - lower to `DW.OP.lit0` followed by popping off the DWARF
stack with `DW.OP.stack_value`
For any remaining unhandled case, we generate `DW.OP.nop` in order
not to mess up remaining DWARF info.
POSIX specifies that the sa_handler field of the sigaction struct may
be set to SIG_IGN or SIG_DFL. However, the current constants in the
standard library use the function pointer signature corresponding to
the sa_sigaction field instead.
This may not cause issues in practice because the fields usually occupy
the same memory in a union, but this isn't required by POSIX and there
may be systems we do not yet support that do this differently.
Fixing this also makes the Zig interface less confusing to use after
reading the man page.
When a local is no longer referenced or used, free it
so the local can be re-used by another instruction.
This means we generate less locals. Freeing this local
is a manual action and must only be used on temporaries
or where we are sure the local is not referenced by a
different AIR instruction, as that creates UB.
We now also no longer store a `WValue` when its tag is set to `none`
as those may never be referenced by any AIR instruction.
An assertion is done to make sure we never store a reference to a
`stack` value in our resolved instructions.
We internally use a lot of `load`'s that used to put
the result in a newly created local. For instance, when is considered
byRef or when we need a specific field/element/bytes from a larger type.
However, sometimes we want to directly use this value and then forget about
it, which means storing it in a local first is wasted instructions as well
as wasted locals that shouldn't be generated in the first place.
With this change it's explicit and requires the usage of `toLocal`.
This also does it for `wrapBinOp` which internally uses the already
refactored `binOp` and `wrapOperand` heavily simplifying this
function and not duplicate the logic from `binOp`
By keeping the result on the stack, we prevent codegen
from generating unneccesary locals when we have subsequent instructions
that do not have to be re-used.
Rather than always creating a new local and storing the result of
a binary operation into said local, we now leave it on top of the stack.
This allows for better codegen as we need less instructions, as well
as less total amount of locals.
When a local is no longer needed (for instance, it was used as
a temporary during arithmetic), it can be appended to one of
the typed freelists. This allows us to re-use locals and therefore
require less locals, reducing the binary size, as well as runtime
initialization.
This effectively reverts 22690efcc2,
re-opening #11818.
This had the following problems:
* Buggy on some targets (macOS, Windows)
* Messy output to the terminal
These problems need to be solved before moving forward with this.
This is likely the cause of the flaky test failures in master branch.
Since we have some test coverage for incremental compilation, it's not
OK to leave proper memory management of Fn objects as "TODO".
This improves the following test case:
```zig
pub fn main() !void {
try foo();
}
fn foo() !void {
return error.Bad;
}
```
The error return trace now points to the `try` token instead of pointing
to the foo() function call, matching stage1.
Closes#12308.
Sema: insert an error return trace frame when appropriate when analyzing
ret_load. Also optimize the instructions to avoid an unnecessary block
sent to the backends.
AstGen: always emit a dbg_stmt for return expressions, in between the
defer instructions and the return instruction.
This improves the following test case:
```zig
pub fn main() !void {
return foo();
}
fn foo() !void {
return error.Bad;
}
```
The error return trace now points to the return token instead of
pointing to the foo() function call, matching stage1.
stage2 was adding bogus error return trace frames when an error was not
being returned. This commit makes several improvements:
* Make a runtime check if necessary to only emit a frame into the error
return trace when an actual error is returned.
* Use the `analyzeIsNonErrComptimeOnly` machinery to avoid runtime
checks when it is compile-time-known that the value is an error, or a
non-error.
* Make std.builtin.returnError take a non-optional stack trace pointer.
closes#12174
This new search placeholder looks much nicer because it allows HTML inside it which
the `placeholder` attribute on `<input>`s doesn't allow.
I tested it for all kinds of cases and it seems to work pretty well.
"Dialog" is the incorrect term here because a dialog is a separate window that still lets you use the app
but a modal is a window where you can't continue using the app until you close it.
- removed an unnecessary (and confusing) `anyerror` fronm the function
signature of `main`
- replaced the call to std.log with two prints: one to stderr and one to
stdout
- replaced the test code with a better example
This way, tracking segment-to-section mapping becomes a lot easier
since it's effectively just start index plus number of sections
defined within the segment. If a section becomes empty however
care needs to be taken to remove the header upon committing to the
final binary.
Fix incorrect writing of symtab and strtab in dSYM bundle in incremental
context.
Fix incorrectly navigating unnamed consts (freeing) in incremental context.
This is currently hard-coded to require all consts to land in `__TEXT,__const`,
which is wrong and needs a rewrite.
Instead of generating sections upfront, allow generation by scanning
the object files for input -> output sections mapping. Next, always
strive to keep output sections in the final container sorted as they
appear in the final binary. This makes the linker less messy wrt
handling of output sections sort order for dyld/macOS not to complain.
There's still more work to be done for incremental context though
to make this work but looks promising already.
Previously we expected to only find decl refs in a `foo.bar.baz`
type of expression. This would crash when trying to render something
like `@typeInfo(T).Int.bits`. We now properly account for builtins
and other components.
As part of the Opaque Pointers upgrade documentation, LLVM says that the
function LLVMGetGEPSourceElementType() can be used to obtain element
type information in lieu of LLVMGetElementType(), however, this function
actually returns the struct type, not the field type. The GEP
instruction does store the information we need, however, this is not
exposed in the C API. It seems like they accidentally exposed the wrong
field, because one would never need the struct type since one must
already pass it directly to the GEP instruction, so one will always have
it handy, whereas one will usually not have the field type handy.
Removed the copy of param_names inside of Fn and changed to
implementation of getParamName to fetch to parameter name from the ZIR.
The signature of getParamName was also changed to take an additional
*Module argument.
From https://man7.org/linux/man-pages/man7/inotify.7.html
> **IN_MASK_CREATE** (since Linux 4.18)
>
> Watch pathname only if it does not already have a watch associated with it; the error EEXIST results if pathname is already being watched.
This change provides a basic implementation of #2349 for stage2. There's
still quite a lot of work before this logic is as complete as what's in
Clang (b364535304/clang/lib/CodeGen/CGStmt.cpp (L2304-L2795)),
particularly considering the diversity of constraints across targets.
It's probably not worth doing the complete work until there's a clearer
picture for constraints in Zig's future dedicated ASM syntax, but at
least this gives us a small improvement for now.
As a bonus, this also fixes a bug with how we were handling `_`
identifiers.
If you have multiple llvm-config executables in your path, and all of
them cause failures, then only the last failure will be printed. This
can cause confusion when the multiple llvm-config executables are from
different major LLVM versions, i.e. LLVM 13 and 14, which might mask an
error that happened on the LLVM 14 llvm-config with an unrelated error.
This commit makes it so that all errors are collected into a list and
printed all at once; this way, you can see how each llvm-config
executable failed to configure properly. Note that the failures still
won't be printed if a successful configuration is found.
We now warn the user if config.h could not be located.
This also updates the search to stop early upon encountering a
`.git` directory, so that we avoid recursing outside of the zig
source if possible.
Fixes: 2a990d696 ("stage1: rework tokenizer to match stage2")
Fixes: b6354ddd5 ("move AST rendering code to separate file")
Signed-off-by: Wei Fu <fuweid89@gmail.com>
This requires using -Dstatic-llvm and setting the search prefix and the
target, just like it is required for building stage2 and stage3. This
prevents Zig from trying to integrate with the system, which would
trigger an error due to the `cc` command not being installed.
closes#12144
Shared libraries can be provided on the command line as if they were
objects, as a path to the ".so" file. The "each-lib-rpath" functionality
was ignoring these shared libraries accidentally, causing missing rpaths
in the output executable.
* No longer emit div_exact AIR instruction that can produce a
remainder, invoking undefined behavior.
* div_trunc, div_exact, div_floor are extracted from analyzeArithmetic
and directly handled similarly to div_trunc, integrating them with
integer overflow safety checking.
* Also they no longer emit divide-by-zero safety checking when RHS
is comptime known to be non-zero.
Concrete improvements:
* Added safety for integer overflow (-MAX_INT/-1)
* Omit division by zero safety check when RHS is comptime known to
be non-zero.
* Avoid emitting `_optimized` variants of AIR instructions for integers
(this suffix is intended to be used for floats only).
Subjective changes: I extracted the div logic out from analyzeArithmetic
in order to reduce the amount of branches - not for performance reasons
but for code clarity. It is more lines of code however, and some logic
is duplicated.
This check is needed because if static/dynamic linking is mixed incorrectly,
it's possible for Clang and LLVM to end up with duplicate "copies" of libc++.
This is not benign: Static variables are not shared, so equality comparisons
that depend on pointers to static variables will fail. One such failure is
std::generic_category(), which causes POSIX error codes to compare as unequal
when passed between LLVM and Clang.
I believe this is the cause of https://github.com/ziglang/zig/issues/11168
In order to avoid affecting build times when Zig is repeatedly invoked,
we only enable this check for "zig env" and "zig version"
Now instead of zig.h being baked into the compiler binary, it is a
header file distributed along with all the other header files
distributed with Zig.
Closes#11643
This causes a stack overflow in a debug build of stage3 unfortunately. I
will open an issue to track this test coverage, which we absolutely
should get working - users of the compiler should get a compile error,
not a segfault if they hit the default branch quota from abusing
recursive inline functions.
Note that the problem does not occur in a release build of stage3
which has significantly reduced stack usage.
On Linux, I tried bumping up the stack size from 32 MiB to 64 MiB and it
did not solve the problem. I'm not sure why not. It seems like it should
be fine.
Note that we also have a problem of running test-cases in multi-threaded
mode which is currently the default. Currently Zig threads are spawned
with 16 MiB stack space.
This makes it so that in a -Dsingle-threaded build of test-cases, if a
crash happens, the test case name will be printed just before the stderr
of the crash.
* proper skip_stage1 mechanism that doesn't get side-stepped with
manually added test cases.
* avoid runtime-known function pointers.
* check for type equality more simply without checking the type name.
At least on Linux, the pwritev syscall checks the pointer and returns
EFAULT before it checks if the length is nonzero.
Perhaps this should be fixed in the standard library, however, these are
still improvements since they make the kernel do less work within the
syscall.
On some systems (esp. systems that use unique hashed file-paths for
library-versions like Nix), we can't expect LLVM and Clang to share
lib/bin directories.
The best we can do is find the matching clang libraries in the
CMAKE_LIBRARY_PATH provided by the environment
`getdents` on Linux can return `ENOENT` if the directory referred to by the fd is deleted during iteration. Returning null when this happens makes sense because:
- `ENOENT` is specific to the Linux implementation of `getdents`
- On other platforms like FreeBSD, `getdents` returns `0` in this scenario, which is functionally equivalent to the `.NOENT => return null` handling on Linux
- In all the usage sites of `Iterator.next` throughout the standard library, translating `ENOENT` returned from `next` as null was the best way to handle it, so the use-case for handling the exact `ENOENT` scenario specifically may not exist to a relevant extent
Previously, ENOENT being returned would trigger `os.unexpectedErrno`.
Closes#12211
When lowering the return type for Wasm if the calling convention is `C`,
it now correctly lower it according to what clang does as specified in:
https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md
This makes use of the same logic as the Wasm backend, ensuring the
generated code does not diverge in function signatures.
When passing arguments accross the C-ABI for the Wasm target,
we want slightly different behavior than x86_64.
For instance: a struct with multiple fields must always be passed
by reference, even if its ABI size fits in a single integer.
However, we do pass larger integers such as 128bit by value,
which LLVM will correctly lower to use double arguments instead.
This reverts commit 7cbd586ace.
This is causing a fail to build from source:
```
./lib/std/fmt.zig:492:17: error: cannot format optional without a specifier (i.e. {?} or {any})
@compileError("cannot format optional without a specifier (i.e. {?} or {any})");
^
./src/link/MachO/Atom.zig:544:26: note: called from here
log.debug(" RELA({s}) @ {x} => %{d} in object({d})", .{
^
```
I looked at the code to fix it but none of those args are optionals.
Expose 2 functions from std.json. These functions take a slice of bytes
and forward them to a given writer as a JSON encoded string.
The use case I have for this is in a custom JsonStringWriter. This writer
takes data and automatically encodes it as JSON string characters and
forwards it to an underlying writer. I use this JsonStringWriter in
combination with std.fmt.format to go directly from a format string/arg
pair to JSON. This way I don't have to format my string into a separate
buffer first and encode it afterwards, which avoids the need to create
a temporary buffer to hold the unencoded but formatted string.
This change relaxes the restriction added in the prior commit that LLD
should be alongside LLVM.
This also leaves unresolved the issue of making sure the link mode
(static or shared) of LLD matches that of LLVM/Clang. That would be an
unfortunate restriction, since LLD seems to be provided only as a static
lib on some distros.
This commit reworks the LLVM/Clang/LLD discovery process for CMake. The
biggest changes are that:
1. We search for LLVM from most preferred directory to least, skipping
any `llvm-config` that is the wrong version, or that doesn't
support the requested link mode ("static" or "shared").
2. `ZIG_PREFER_CLANG_CPP_DYLIB` has been renamed to `ZIG_SHARED_LLVM`,
to better align with `ZIG_STATIC_LLVM`.
3. We only search for LLVM in the same directory alongside LLVM.
4. LLVM's link mode is forwarded to Clang, so that we can look for the
appropriate shared/static libraries.
5. We use `--link-static` when querying `--system-libs` from llvm-config,
so that this will include libz and other dependencies for
statically linking LLD
The current phrasing is vague; it is unclear whether it is demonstrating an example of the type of permitted behavior, from which the rule set must be extrapolated, or it is stating that this restriction only applies to the relationship between integers and bare structs.
Implements running and verifying the expected output when a binary is run.
Also adds warnings when a binary is skipped because of incompatibility.
This warning can be hidden by either setting the option manually through build.zig,
or by providing the option `-Dhide_foreign_warnings`.
This creates a new step that can run foreign binaries when
emulation is enabled using options such as `enable_qemu`.
When an incompatible binary is found, the binary will not be executed.
This differs from `RunStep` which will always execute a binary,
regardless of the compatibility.
This is useful for usecases where the user wishes to allow for running the
binary on any supported platform either natively or through emulation,
but not generate an error when met with an incompatibility.
The above is useful when creating test cases that rely on running the binary
and optionally verifying its output.
The addition of this Step was generated by the need for our linker tests.
For that reason, a handy function was created on `CheckObjectStep` to ease
the setup for that.
Previously, we would get a pointer to a slot in the symbol table,
apply changes to the symbol, and return the pointer. This however
didn't take into account that the symbol table may be moved in memory
in-between the modification and return from the function (`fn placeDecl`).
Prior to my rewrite, this was not possible within the body of the said
function. However, my rewrite revamped how we allocate GOT atoms and
their matching symtab indexes, which now may cause a move in memory
of the container.
`/usr/local/include`, `/usr/local/lib` and `/Library/Frameworks`
have been deprecated since approximately macOS 11, and so to avoid
redundant and misinformed warning messages generated by the linker,
add those dirs only when natively targeting macOS 10.x.x or below.
update() calls mem.indexOfScalar() which uses `==` for comparing items,
which fails when the operator is not supported.
As PirorityQueue needs a comparing function anyway we can use `.eq` results
to identify matching objects.
closes#9918
architectures
The idea behind this is using the register capabilities in safe amounts,
there is still some consideration to be done.
+ Fixed compile error using std.Target.<arch>.featureSetHas
+ X86 MMX and "3DNOW" 64 bits register usage considered for vector size
+ Added ARM Neon recommened usage of 128 bits (The size of the register)
+ Added AARCH64 Neon and SVE for 128 bits. SVE could use in theory up to
2048 bits, but found only evidence of functional 512 bits on a super
computer, decided on using 128 bits as a safety
+ Added Altivec recommendation of using the 128 bits long register
+ Using MIPS msa 2x64bits capabilities, usage of 64 bits registers for MDMX
systems, need testing on how using bigger values affect performance
+ Using V extension on RISC-V, which is extendable via instructions, decided on 128 bits
as a value to not use all registers
+ in SPARC the 64 bits registers are used, a max of 32 registers
are to be used for configurable simd instructions, decided on using
the size of the register, need testing on performance hit on using a
bigger sized register vector size
In x86_64 relocs, it can so happen that the compiler
refers to the same atom by both the actual assigned symbol
and the start of the section. In this case, we need to
link the two together so add an alias.
Now, each object file will store a mutable table of symbols that it
defines. Upon symbol resolution between object files, the symbol
will be updated with a globally allocated section ordinal and address
in virtual memory. If the object defines a globally available symbol,
its location only (comprising of the symbol index and object index)
will be stored in the globals map for easy access when relocating, etc.
This approach cleans up the symbol management significantly, and matches
the status quo used in zld/ELF.
Additionally, this makes scoping symbol stabs easier too as they are
now naturally contained within each object file.
When lowering a struct type to an LLVM struct type, keep track of
whether there are any underaligned fields. If so, then make it a packed
llvm struct. This works because we already insert manual padding bytes
regardless.
We could unconditionally use an LLVM packed struct; the reason we bother
checking for underaligned fields is that it is a conservative choice, in
case LLVM handles packed structs less optimally. A future improvement
could simplify this code by unconditionally using packed LLVM structs
and then make sure measure perf is unaffected.
closes#12190
`validateExternType` does not require the type to be resolved so we can
check it earlier. Only doing it in `resolveTypeFully` lead to worse or
missing compile errors.
* Use a debug build of stage3 instead of a debug build of stage2 for
our self-hosted compiler test coverage.
* Move coverage from stage1 to stage3 for:
- building self-hosted without LLVM
- building self-hosted for 32-bit arm
- test-compiler-rt
- test-behavior
- test-std
- test-compare-output
- test-asm-link
- test-fmt
We were accidentally returning a pointer to stack memory, because these
arguments were passed by value. It's just an accident that stage 1 was
passing these by reference, so things were alright until stage 3.
We're now using `std.json.writeStream`, which makes our prints correct
in terms of escapes and also reduces the amount of json-related code.
Unfortunately, we have to mess around with the json stream writer state
whenever we end up using `std.json.stringify` for convenience.
originally I thought `foo.bar.baz` was a path of decls, but turns out
other language constructs require to make this model more general.
originally a decl path was an array of decl indexes, now it's an array
of `WalkResult`s
while it would be preferable to not save decltests as first-class decls
(and just embed their information inside the decl they refer), adding
logic to skip them complicates the code too much so we should consider
this an optimization for the future.
This is a possible workaround for
https://github.com/llvm/llvm-project/issues/56585
On my computer it makes stage3-release go from false positive
compilation errors on the behavior tests to "segmentation fault".
Is this forwards progress or backwards progress? I have no idea.
See #11450
Since we know the offset, we may as well read starting there. Still expects
rpath to fit in 4096 bytes; that might be worth fixing in the future.
Fixes issue #12112
This is a workaround for https://github.com/llvm/llvm-project/issues/56585
which causes writes to i1 in memory to be optimized to an incorrect value.
Unfortunately, this does not save users from running into this bug with u1
in their own types.
However, this does seem to be enough to get the behavior tests working.
This resolves#11450 on my machine.
* io_uring: fix the timeout_remove test
The test does a IORING_OP_TIMEOUT followed with a IORING_OP_TIMEOUT_REMOVE
and assumed we would get the CQEs in the same order.
Linux v5.18 changed how this works and we now get them in the reverse order.
The documentation doesn't explicitly say which CQE we should get first
so just make the test work with both cases.
* io_uring: fix the remove_buffers test
The original test was buggy but accidentally worked with kernels < 5.18
The test assumed that IORING_OP_REMOVE_BUFFERS removed from the start of
but in fact the documentation doesn't specify which buffer is removed,
only that a certain number of buffers are removed.
Starting with the kernel 5.18 the check for the `used_buffer_id` fails.
Turns out that previous kernels removed buffers in such a way that the
remaining buffer for this read would always be 0, however this isn't
true anymore.
Instead of checking a specific value just check that the `used_buffer_id`
corresponds to a valid ID.
Before this would fail to compile:
```
fn testFn(alloc: std.mem.Allocator, arr: []const u8) !void {
_ = alloc;
_ = arr;
}
test "checkAll" {
var arr = [_]u8{ 1, 2, 3 };
try std.testing.checkAllAllocationFailures(std.testing.allocator, testFn, .{arr[0..]});
}
```
with the error `error: Unexpected type for extra argument at index 0: expected []const u8, found *[3]u8`
By removing this strict equality check, we allow the type checking to be done during the `@field(args, arg_i_str) = @field(extra_args, field.name);` instead, which then allows for things like type coercion to work, but still will give a compile error if the types are incorrect. So, after this change, the above succeeds (because `*[3]u8` can be coerced to `[]const u8`).
The new compile error when providing an incorrect type that can't be coerced looks like this:
```
zig/lib/std/testing.zig:639:35: error: expected type '[]const u8', found '*[3]u32'
@field(args, arg_i_str) = @field(extra_args, field.name);
^
```
This is really minor but the issue this fixes is that if you copy-paste this output of `--show-builtin` into your `build.zig` for example then the formatter will format
```
pub const os = std.Target.Os{
.tag = .freestanding,
.version_range = .{ .none = {} }
};
```
to
```
pub const os = std.Target.Os{ .tag = .freestanding, .version_range = .{ .none = {} } };
```
which doesn't match the output.
With this comma, the output will stay the way it is after a `zig fmt`.
This implementation uses the F_KINFO fcntl command added in FreeBSD
13 release. FreeBSD 12 users get a compile error.
Co-authored-by: Stephen Gregoratto <dev@sgregoratto.me>
Previously, the Zig ABI size and LLVM ABI size of these types disagreed
sometimes. This code also corrects the logging messages to not trigger
LLVM assertions.
This reverts commit 2eaef84ebe.
Here is a motivating example:
```zig
const E = union(enum) {
A: [9]u8,
B: u64,
};
```
```llvm
%test2.E = type { { i64, [1 x i8] }, i1, [6 x i8] }
```
```
error(codegen): when lowering test2.E, Zig ABI size = 16 but LLVM ABI size = 24
```
CMake has a surprising default behavior where looking up a library by
multiple names gives the name order higher priority than the directory
search order.
For example, if your system provides "llvm-config-14" and
CMAKE_PREFIX_PATH includes "llvm-config", CMake will always end up
choosing the system-provided llvm-config-14.
This change add NAMES_PER_DIR to request the more sensible behavior:
directory search order has higher priority than name order, so
CMAKE_PREFIX_PATH always wins over system-provided tools/libraries.
It did not handle properly when the dummy operand was a comptime_int; it
was crashing in coerce because comptime_int is supposed to be
comptime-known. So when calling coerceResultPtr, we pass the actual
operand, not a dummy operand, which means it will have the proper
comptime value when necessary.
Previously, the logic for analyzing coerce_result_ptr would generate
invalid bitcast instructions which did not include coercion logic, such
as optional wrapping, resulting in miscompilations.
Now, the logic of resolve_inferred_alloc goes back over all the
placeholders inserted by coerce_result_ptr, and replaces them with logic
doing the proper coercions.
Closes#12045
* Sema: implement comptime bitcast of f80 with integer-like types
bitwise rather than taking a round trip through memory layout.
* Type: introduce `isAbiInt`.
* Value: comptime memory write of f80 writes 0 bytes for padding
instead of leaving the memory uninitialized.
* Value: floatReadFromMemory has a more general implementation, checking
the endianness rather than checking for specific architectures.
This fixes behavior test failures occurring on MIPS.
For calling convention ABI purposes, integer attributes and return
values need to have an LLVM attribute signext or zeroext added
sometimes. This commit implements that logic.
It also implements a proof-of-concept of moving the F16T type from
being a compiler_rt hack to being how the compiler lowers f16 in
functions that need to match certain calling conventions.
Closes#12054
These functions semantically benefit from being inline; it makes sense
that `isInf(x)` where `x` is comptime-known should have a
comptime-known result.
Previously, struct types, alignment values, and initialization
expressions were all lowered into the same ZIR body, which caused false
positive "depends on itself" errors when the initialization expression
depended on the size of the struct.
This also uses ResultLoc.coerced_ty for struct field alignment and
initialization values. The resulting ZIR encoding ends up being roughly
the same, neither smaller nor larger than previously.
Closes#12029
For some targets, Clang unconditionally adds some clobbers to all inline assembly.
While this is probably not strictly necessary, if we don't follow Clang's lead
here then we may risk tripping LLVM bugs since anything not used by Clang tends
to be buggy and regress often.
LLVM does not properly handle debug info for f16 on the aarch64-windows
target, causing "fatal error: unknown codeview register H1". The
previous workaround checked only for f16 but was still vulnerable if a
type was a byval struct or tuple which had an f16 field in it.
Now I have filed an upstream issue (see
https://github.com/llvm/llvm-project/issues/56484) and broadened the
workaround to always skip debug values for this target.
Our lowerings for various LLVM types assume that we can anticipate the
alignment/layout that LLVM will generate. Among other things, this
requires that we keep the alignment of our lowered LLVM types
synchronized with their expected alignment in Zig.
- Arrays were using packed struct types, which is seems to be
incorrect since array elements are supposed to be self-aligned.
- Unions were using packed struct types for their payload, which causes
layout divergence between what stage2 expects and what LLVM generates
Consider this lowered union type:
```llvm
%Value = type { <{ i64, [8 x i8] }>, i1, [7 x i8] } ; 24 bytes, align(1)
%ErrorUnion = type { %Value, i16 } ; 26 bytes, align(2)
```
Zig expects Value to be align(8) and, by extension, for ErrorUnion to be
size 32.
* test/link: initial wasm support
This adds basic parsing and dumping of wasm section so they
can be tested using the new linker-test infrastructure.
* test/link: all wasm sections parsing and dumping
We now parse and dump all sections for the wasm binary format.
Currently, this only dumps the name of a custom section.
Later this should also dump symbol table, name, linking metadata and relocations.
All of those live within the custom sections.
* Add wasm linker test
This also fixes a parser mistake in reading the flags.
* test/link: implement linker tests wasm & fixes
Adds several test cases to test the wasm self-hosted linker.
This also introduces fixes that were caught during the implementation
of those tests.
* test-runner: obey omit_stage2 for standalone
When a standalone test requires stage2, but stage2 is omit
from the compiler, such test case will not be included as part
of the test suite that is being ran. This is to support CI's
where we omit stage2 to lower the memory usage.
* make the setting in the linker backend be non-optional; by this time
all defaults are supposed to be resolved.
* integrate with `zig cc`
* change the CLI parsing to match C compiler parsing, allowing
`--compress-debug-sections` alone to choose a default encoding of
zlib.
Those 6 sets of square brackets are just a typographical aid used in this doc-comment, and must not actually be written by the user in their own format string... except for in one case where they must
* Avoid the implication that the numeric index is 1-based rather than 0-based
This change is the Zig counterpart to https://reviews.llvm.org/D110413
Since we lower some libcalls directly (just like clang does), we need to
make sure that the ABI we call with matches the ABI of the compiler-rt
we are providing (and also the ABI expected by LLVM).
While I was at it, I noticed some flawed vector handling in the binary
soft float ops in stage 1, so I shored up the logic a bit and expanded
an existing test to cover the missing functionality.
Starting with LLVM 14, the Libcalls to these functions are now lowered
using a Vec(2, u64) instead of the standard ABI for i128 integers, so
our compiler-rt implementation needs to be updated to expose the same
ABI on Windows.
This is intended to fix this error:
lld-link: error: undefined symbol: _GUID const& __mingw_uuidof<IEnumSetupInstances>()
>>> referenced by D:\a\1\s\src\windows_com.hpp:898
>>> D:\a\1\s\zig-cache\o\a5030d467932f0ce2f6511feb7d6af12\windows_sdk.obj:(__IEnumSetupInstances_IID_getter())
Thank you to Martin Storsjö for suggesting the fix.
This commit adds support for initializing `.anon_struct` types. There
is also some follow-up work to do for both tuples and structs regarding
comptime fields, so this also adds some tests to keep track of that
work.
Key blinding allows public keys to be augmented with a secret
scalar, making multiple signatures from the same signer unlinkable.
https://datatracker.ietf.org/doc/draft-dew-cfrg-signature-key-blinding/
This is required by privacy-preserving applications such as Tor
onion services and the PrivacyPass protocol.
The previous code here was potentially more optimal for some cases,
however, I never tested the perf, so it might not actually matter. This
code handles more cases. We can go back and re-evaluate that other
implementation if it seems worthwhile in the future.
This accomplishes two things:
* Works around #8442 by putting stage1-specific logic in to disable all
the std.json tests.
* Slightly reduces installation size of zig since std lib files ending
in "test.zig" are excluded from being installed.
Commit 3014a0d5f1 added calling convention
validation, but left out thumb and thumbeb from the archs that can use the
ARM-specific calling conventions. This plus
a8a7f15106 made compilation fail for thumb
targets.
The fstype argument to the mount system call can be null. To see an
example run "strace -e trace=mount unshare -m":
```
mount("none", "/", NULL, MS_REC|MS_PRIVATE, NULL) = 0
...
```
Future improvement: make plain error notes actually render as notes
rather than errors, but keep them as errors for the case of
sub-compilation errors, e.g. when compiler-rt has compilation errors.
The previous definition depends on a non-lang-spec-compliant memory
layout for packed structs, which happens to trigger #11989 in stage2.
This commit changes the struct to be an extern struct with an
align(4) field. However, stage1 cannot handle this, so conditional
compilation logic is used to select different struct definitions
depending on stage1 vs stage2.
This works around #11989 but does not solve the underlying problem -
putting an extern union inside a packed struct will still trigger the
assert.
After this, both stage1 and stage2 std lib tests run assertion-clean
with a debug LLVM 13.
This follows LLVM14's lead on vector alignment, which computes byte
count based on the length premultiplied by bits.
This commit also disables behavior tests regressed by LLVM 14, only for
stage1. stage2 fortunately does not trip the regression.
LLVM 14 makes it so that a RHS of saturating shift left produces a
poison value if the value is greater than the number of bits of the LHS.
Zig now emits code that will check if this is the case and select a
saturated LHS value in such case, matching Zig semantics.
Notable changes:
`_i386`, `_i486`, and `_i686` are renamed to `i386`, `i486`,
and `i686` respectively. `std.zig.fmtId` is enhanced to support
formatting `i386` as `@"i386"`.
Some CPU features which are actually CPU models have been
properly flattened, such as `apple_a12`, `apple_a13`, `apple_a7`,
`cortex_a78c`, `exynos_m4`, `neoverse_e1`, `neoverse_n1`,
`neoverse_n2`, `neoverse_v1`.
Some CPU features have been added and some have been removed, following
LLVM's lead.
CSky CPU features support is added.
LLVM pointers are transitioning to no longer have types, however, inline
assembly inputs and outputs which accept pointers need to know the
element type. So, inline assembly must be upgraded to add
elementtype(<ty>) annotations.
LLVM 14 deprecated a bunch of C API functions in preparation for opaque
pointer changes. However, they did not actually implement opaque pointer
semantics, so the deprecations are largely masturbatory. I have nothing
against masturbation, I am just busy trying to get the self hosted
compiler done for 0.10.0, so we will come back to this later.
This is a prelude to a more elaborate work which will implement
`-dead_strip` flag - garbage collection of unreachable atoms. Here,
when sorting sections, we also check that the section is actually
populated with some atoms, and if not, we exclude it from the final
linked image. This can happen when we do not import any symbols
from dynamic libraries in which case we will not be populating
the stubs sections or the GOT table, implying we can skip allocating
those sections. Furthermore, we also make a check that a segment
is actually occupied too, with the exception of `__TEXT` segment
which is non-optional given that it wraps the header and load commands
and thus is required by the `dyld` to perform dynamic linking, and
`__PAGEZERO` which is generally non-optional when the linked image
is an executable. For any other segment, if its section count is
zero, we mark it as dead and skip allocating it and generating
a load command for it.
This commit also includes some minor improvements to the linker such
as refactoring of the segment allocating codepaths, skipping
`__PAGEZERO` generation for dylibs, and skipping generation of zero-sized
atoms for special symbols such as `__mh_execute_header` and `___dso_handle`.
These special symbols are only allocated local and global symbol pair
and their VM addresses is set to the start of the `__TEXT` segment,
but no `Atom` is created, as it's not necessary given that they never
carry any machine code.
Finally, we now always force-link against `libSystem` which turns out
to be required for `dyld` to properly handle `LC_MAIN` load command
on older macOS versions such as 10.15.7.
Gimli was a game changer. A permutation that is large enough to be
used in sponge-like constructions, yet small enough to be compact
to implement and fast on a wide range of platforms.
And Gimli being part of the Zig standard library was awesome.
But since then, Gimli entered the NIST Lightweight Cryptography
Competition, competing againt other candidates sharing a similar set
of properties.
Unfortunately, Gimli didn't pass the 3rd round.
There are no practical attacks against Gimli when used correctly, but
NIST's decision means that Gimli is unlikely to ever get any traction.
So, maybe the time has come to move Gimli from the standard library
to another repository.
We shouldn't do it without providing an alternative, though.
And the best candidate for this is probably Xoodoo.
Xoodoo is the core function of Xoodyak, one of the finalists of the
NIST LWC competition, and the most direct competitor to Gimli. It is
also a 384-bit permutation, so it can easily be used everywhere Gimli
was used with no parameter changes.
It is the building block of Xoodyak (for actual encryption and hashing)
as well as Charm, that some Zig applications are already using.
Like Gimli that it was heavily inspired from, it is compact and
suitable for constrained environments.
This change adds the Xoodoo permutation to std.crypto.core.
The set of public functions includes everything required to later
implement existing Xoodoo-based constructions.
In order to prepare for the Gimli deprecation, the default
CSPRNG was changed to a Xoodoo-based that works exactly the same way.
A hash function cascade was a common way to avoid length-extension
attacks with traditional hash functions such as the SHA-2 family.
Add `std.crypto.hash.composition` to do exactly that using arbitrary
hash functions, and pre-define the common SHA2-based ones.
With this, we can now sign and verify Bitcoin signatures in pure Zig.
The constant value lowering for unions was missing a check for whether
the payload was itself an unnamed struct. Lowerings of other types
already handle this case.
closes#11971
This regressed; the plan9 linker code is crashing when trying to build
compiler-rt and we have no active plan9 maintainers involved in the Zig
project. Anyone is welcome to come over and take the role; however, it's
not one of the tier 1, 2, or 3 targets, so we will not be blocking
progress towards 1.0 on plan9.
Zig guarantees the memory layout of f16, f32, f64, f80, and f128 which
means for generic function purposes, values of these types need to be
compared on the basis of their bits in memory. This means nan-packing
can be used with generic functions, for example.
For comptime_float, the sign is observable, whether it is nan is
observable, but not any more kinds of bit patterns are observable.
This fixes the std.fmt tests that check printing "-nan".
Rather than lowering float negation as `0.0 - x`.
* Add AIR instruction for float negation.
* Add compiler-rt functions for f128, f80 negation
closes#11853
The LLVM backend was calculating the amount of padding solely based
on the payload size. However, in the case where there is no union
tag, this fails to take into account alignment.
Closes#11857
Many of the Managed methods accepted by-val parameters which could
reference Limb slices that became invalid memory after any
ensureCapacity calls. Now, Managed methods accept `*const Managed`
parameters so that if the function allows aliasing and the
ensure-capacity call resizes the Limb slice, it also affects the
aliased parameters, avoiding use-after-free bugs.
This is a breaking change that reduces the requirement for callsites to
manually make the ensure-capacity changes prior to calling many of the
Managed methods.
Closes#11897
std.crypto.ecc: add support for the secp256k1 curve
Usage of the secp256k1 elliptic curve recently grew exponentially,
since this is the curve used by Bitcoin and other popular blockchains
such as Ethereum.
With this, Zig has support for all the widely deployed elliptic curves
today.
* mem: refactor tests of split()
- add a few cases for .rest()
- use expectEqualSlices()
* mem: add splitBackwards
Over the last couple of weeks weeks I needed to iterate over a
collection backwards at least twice. Do we want to have this in stdlib?
If yes, click "Merge" and start using today! Free shipping and returns
(before 1.0).
Why is this useful?
-------------------
I need this for building an error wrapper: errors are added in the
wrapper from "lowest" level to "highest" level, and then printed in
reverse order. Imagine `UpdateUsers` call, which needs to return
`error.InvalidInput` and a wrappable error context. In Go we would add a
context to the error when returning it:
// if update_user fails, add context on which user we are operating
if err := update_user(user); err != nil {
return fmt.Errorf("user id=%d: %w", user.id, err)
}
Since Zig cannot pass anything else than u16 with an error (#2647), I
will pass a `err_ctx: *Err`, to the callers, where they can, besides
returning an error, augment it with auxiliary data. `Err` is a
preallocated array that can add zero-byte-separated strings. For a
concrete example, imagine such a call graph:
update_user(User, *Err) error{InvalidInput}!<...>
validate_user([]const u8, *Err) error{InvalidInput}!<...>
Where `validate_user` would like, besides only the error, signal the
invalid field. And `update_user`, besides the error, would signal the
offending user id.
We also don't want the low-level functions to know in which context they
are operating to construct a meaningful error message: if validation
fails, they append their "context" to the buffer. To translate/augment
the Go example above:
pub fn validate_user(err_ctx: *Err, user: User) error{InvalidInput}!void {
const name = user.name;
if (!ascii.isAlpha(name)) {
err_ctx.print("name '{s}' must be ascii-letters only", .{name});
return error.InvalidInput;
}
<...>
}
// update_user validates each user and does something with it.
pub fn update_user(err_ctx: *Err, user: User) error{InvalidInput}!void {
// validate the user before updating it
validate_user(user) catch {
err_ctx.print("user id={d}", .{user.id});
return error.InvalidInput;
};
<...>
}
Then the top-level function (in my case, CLI) will read the buffer
backwards (splitting on `"\x00"`) and print:
user id=123: name 'Žvangalas' must be ascii-letters only
To read that buffer backwards, dear readers of this commit message, I
need `mem.splitBackwards`.
For 25519, it's very likely that applications would ever need the
serialized representation. Expose the value as an integer as in
other curves. Rename the internal representation from `field_size`
to `field_order` for consistency.
Also fix a common typo in `scalar.sub()`.
This reverts commit 8bf3e1f8d0, which
introduced miscompilations for peer expressions any time they needed
coercions to runtime types.
I opened #11957 as a proposal to accomplish the goal of the reverted
commit.
Closes#11898
Improve testing MachO binaries by verbose printing of the symtab
which includes segment,section names for defined symbols, and
import (dylib) name for imports.
This valid zig code produces reasonable LLVM IR, however, on the
wasm32-wasi target, when using the wasmtime runtime, the number of
locals of the `isSquare` function exceeds 50000, causing wasmtime
to refuse to execute the binary.
The `inline` keyword in Zig is intended to be used only where it is
semantically necessary; not as an optimization hint. Otherwise, this may
produce unwanted binary bloat for the -OReleaseSmall use case.
In the future, it is possible that we may end up with both `inline`
keyword, which operates as it does in status quo, and additionally
`callconv(.inline_hint)` which has no semantic impact, but may be
observed by optimization passes.
In this commit, I also cleaned up `isSquare` by eliminating an
unnecessary mutable variable, replacing it with several local constants.
Closes#11947.
MachO linker now handles `-needed-l<name>`, `-needed_library=<name>`
and `-needed_framework=<name>`. While on macOS `-l` is equivalent
to `-needed-l`, and `-framework` to `-needed_framework`, it can be
used to the same effect as on Linux if combined with `-dead_strip_dylibs`.
This commit also adds handling for `-needed_library` which is macOS
specific flag only (in addition to `-needed-l`).
Finally, in order to leverage new linker testing harness, this commit
added ability to specify lowering to those flags via `build.zig`:
`linkSystemLibraryNeeded` (and related), and `linkFrameworkNeeded`.
Before this commit, the passed in length would always be given to the RtlCaptureStackBackTrace call. Now we always give the length of the actual buffer we're using (the addr_buf_stack size of 32 or the passed in length if it's larger than 32; this matches what the doc comment says the function was meant to be doing as well).
This was causing empty stack traces for things like the GeneralPurposeAllocator leak checking.
Fixes#6687
We now ensure the "bss" section is last, which allows us to not
emit this section and let the runtime initialize the memory with 0's instead.
This allows for smaller binaries.
The order of the other segments is arbitrary and does not matter, this may
change in the future.
Includes both traditiona and incremental codepaths with one caveat that
in incremental case, the requested size cannot be smaller than the
default padding size due to prealloc required due to incremental nature
of linking.
Also parse `-headerpad_max_install_names`, however, not actionable just yet -
missing implementation.
Unlike targeting ELF-based OSes such as Linux, resolving system libs
on Darwin should follow one of two strategies: `-search_paths_first`
or `-search_dylibs_first` and hence we defer always forcing linking
a static library to the linker.
Decls will now be put into their respective segment.
e.g. a constant decl will be inserted into the "rodata" segment,
whereas an uninitialized decl will be put in the "bss" segment instead.
Ignore MachO-specific flag -search_paths_first, since it is the default
in zld and ld64.
Also see Jakub's comment[1]:
Changing topic slightly, @motiejus dunno if you noticed, with this
change building arm64 Zig binary fails on macos - it complains about
unknown -search_paths_first flag. This one is an easy fix: we should
ignore it since this is the default behaviour in both ld64 and zld.
[1]: https://github.com/ziglang/zig/pull/11906#issuecomment-1163545849
Change BigTomb implementation to call to Liveness' implementation
rather than implementing feed() in itself.
This is modelled after the AArch64 backend.
This adds clarification to the getGlobalSymbol doc comments,
as well as renames the `addExternFn` function for MachO to `getGlobalSymbol`.
This function will now be called from 'src/link.zig' as well.
Finally, this also enables compiling zig's libc using LLVM even though
the `fno-LLVM` flag is given.
`genFunctype` now accepts calling convention, param types, and return type
as part of its function signature rather than `fnData`. This means
we no longer have to create a dummy for our intrinsic call abstraction.
This also adds support for f16 division and builtins such as `@ceil` & more.
Rather than checking if the user wants to use LLVM for the current compilation,
check for the existance of LLVM as part of the compiler. This is temporarily,
until other backends gain the ability to compiler LLVM themselves.
This means that when a user passed `-fno-LLVM` we will use the native
backend for the user's code, but use LLVM for compiler-rt.
This also fixes emitting names for symbols in the Wasm linker,
by deduplicating symbol names when multiple symbols point the same object.
Implements `@mulAdd` for floats with bitsize 16, where it generates
a call into compiler-rt's `fmaf` function. Note that arguments
for fmaf are different in order than `@mulAdd`.
This implements binary operations and comparisons
for floats with bitsize 16. It does this by calling into
compiler-rt to first extend the float to 32 bits, perform the operation,
and then finally truncate back to 16 bits. When loading and storing the f16,
we do this as an unsigned 16bit integer.
Rather than finding the original object file, we seekTo to the
object file's position within the archive file, and from there open
a new file handle. This file handle is passed to the `Object` parser
which will create the object.
When a new symbol is resolved to an existing symbol where
it doesn't overwrite the existing symbol, we now add this symbol
to the discarded list. This is required so when any relocation points
to the symbol, we can retrieve the correct symbol it's resolved by instead.
When performing relocations for a type index,
we first check if the target symbol is undefined. In which case,
we will obtain the type from the `import` rather than look into the
`functions` table.
This implements a very basic archive file parser that validates
the magic bytes, and then parses the symbol table and stores
the symbol and their position.
Multiple symbols can point to the same function, this means that when we loop over
the symbol list, we must deduplicate those functions being added twice.
Additionaly, we must also ensure that when we append a new type and set the type
index on a function, we must not do this again for the same function.
This commit also implements sorting of code atoms to ensure their order matches
the order of the function section to ensure the function signature matches
that of the function body.
Implements the creation of an undefined symbol for a compiler-rt intrinsic.
Also implements the building of the function call to said compiler-rt intrinsic.
Inducing failure but not getting OutOfMemory back is not as much of a problem as never inducing failure when it was expected to be induced, so treating them differently and allowing them to be handled differently by the caller is useful.
For example, the current implementation of `std.HashMapUnmanaged.getOrPutContextAdapted` always tries to grow and then recovers from OutOfMemory by attempting a lookup of an existing key. If this function is used (i.e. from `std.BufMap.putMove`) with `checkAllAllocationFailures`, then we'd have previously triggered `error.NondeterministicMemoryUsage`, but the real cause is that `OutOfMemory` is being recovered from and so the error is being swallowed. The new error allows us to both understand what's happening easier and to catch it and ignore it if we're okay with the code we're testing handling `error.OutOfMemory` without always bubbling it up.
The for-loop in dump() would index out of bounds if `t.index` is greater
than size, because `end` is the maximum of `t.index` and `size` rather than the
minimum.
This approach is more inline with what LLVM/LLD does for testing
of their output, and seems to be more generic and easier to extend
than implementing a lot of repetitive and nontrivial comparison
logic when working directly on structures.
CheckMachOStep specialises CheckFileStep into directed (surgical)
MachO file fuzzy searches. This will be the building block for
comprehensive MachO linker tests.
If page aligned requested pagezero size is 0, skip generating
__PAGEZERO segment.
Add misc improvements to the pipeline, and correctly transfer the
requested __PAGEZERO size to the linker.
Pass `-pagezero_size` to the MachO linker. This is the final
"unsupported linker arg" that I could chase that CGo uses. After this
and #11874 we may be able to fail on an "unsupported linker arg" instead
of emiting a warning.
Test case:
zig=/code/zig/build/zig
CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 CC="$zig cc -target x86_64-macos" CXX="$zig c++ -target x86_64-macos" go build -a -ldflags "-s -w" cgo.go
I compiled a trivial CGo program and executed it on an amd64 Darwin
host.
To be honest, I am not entirely sure what this is doing. This feels
right after reading what this argument does in LLVM sources, but I am by
no means qualified to make MachO pull requests. Will take feedback.
wasm32-wasi-musl wants the standard symbol names however Linux requires
the `__gnu_*` flavors. I did not find any authoritative source on what
decides which symbol flavors to use. If we run into more trouble in the
future we can go back to having both.
After doing performance testing, it seems that multi-compilation-unit
compiler-rt did not bring the performance improvements that we expected
it to. The idea is that it makes linking faster, however, it incurred a
cost in the frontend that was not offset by any gains in linking.
Furthermore, the single-object compiler-rt (with -ffunction-sections and
--gc-sections) ends up being fewer bytes on disk and so it's actually
the same or faster linking speed than the multi-compilation-unit
version.
So we are planning to keep using single-compilation-unit compiler-rt for
the foreseeable future, but may experiment with this again in the
future, in which case this commit can be reverted.
Finishes cleanups that I started in other commits in this branch.
* Use common.linkage for all exports instead of redoing the logic in
each file.
* Remove pointless `@setRuntimeSafety` calls.
* Avoid redundantly exporting multiple versions of functions. For
example, if PPC wants `ceilf128` then don't also export `ceilq`;
similarly if ARM wants `__aeabi_ddiv` then don't also export
`__divdf3`.
* Use `inline` for helper functions instead of making inline calls at
callsites.
compiler_rt_lib and compiler_rt_obj are extracted from the generic
JobQueue into simple boolean flags, and then handled explicitly inside
performAllTheWork().
Introduced generic handling of allocation failure and made
setMiscFailure not return a possible error.
Building the compiler-rt static library now takes advantage of
Compilation's ThreadPool. This introduced a problem, however, because
now each of the object files of compiler-rt all perform AstGen for the
full standard library and compiler-rt files. Even though all of them end
up being cache hits except for the first ones, this is wasteful - O(N*M)
where N is number of compilation units inside compiler-rt and M is the
number of .zig files in the standard library and compiler-rt combined.
More importantly, however, it causes a deadlock, because each thread
interacts with a file system lock for doing AstGen on files, and threads
end up waiting for each other. This will need to be handled with a
process-level file caching system, or some other creative solution.
The purpose of this branch is to switch to using an object file for each
independent function, in order to make linking simpler - instead of
relying on `-ffunction-sections` and `--gc-sections`, which involves the
linker doing the work of linking everything and then undoing work via
garbage collection, this will allow the linker to only include the
compilation units that are depended on in the first place.
This commit makes progress towards that goal.
This passes -Wl,-no-pie linker arg. Golang uses that. From the `ld(1)`
man page:
Create a position dependent executable. This is the default.
Not adding to the help text, because this is the default.
ECDSA is the most commonly used signature scheme today, mainly for
historical and conformance reasons. It is a necessary evil for
many standard protocols such as TLS and JWT.
It is tricky to implement securely and has been the root cause of
multiple security disasters, from the Playstation 3 hack to multiple
critical issues in OpenSSL and Java.
This implementation combines lessons learned from the past with
recent recommendations.
In Zig, the NIST curves that ECDSA is almost always instantied with
use formally verified field arithmetic, giving us peace of mind
even on edge cases. And the API rejects neutral elements where it
matters, and unconditionally checks for non-canonical encoding for
scalars and group elements. This automatically eliminates common
vulnerabilities such as https://sk.tl/2LpS695v .
ECDSA's security heavily relies on the security of the random number
generator, which is a concern in some environments.
This implementation mitigates this by computing deterministic
nonces using the conservative scheme from Pornin et al. with the
optional addition of randomness as proposed in Ericsson's
"Deterministic ECDSA and EdDSA Signatures with Additional Randomness"
document. This approach mitigates both the implications of a weak RNG
and the practical implications of fault attacks.
Project Wycheproof is a Google project to test crypto libraries against
known attacks by triggering edge cases. It discovered vulnerabilities
in virtually all major ECDSA implementations.
The entire set of ECDSA-P256-SHA256 test vectors from Project Wycheproof
is included here. Zero defects were found in this implementation.
The public API differs from the Ed25519 one. Instead of raw byte strings
for keys and signatures, we introduce Signature, PublicKey and SecretKey
structures.
The reason is that a raw byte representation would not be optimal.
There are multiple standard representations for keys and signatures,
and decoding/encoding them may not be cheap (field elements have to be
converted from/to the montgomery domain).
So, the intent is to eventually move ed25519 to the same API, which
is not going to introduce any performance regression, but will bring
us a consistent API, that we can also reuse for RSA.
Instead of always using std.testing.allocator, the test harness now follows
the same logic as self-hosted for choosing an allocator - that is - it
uses C allocator when linking libc, std.testing.allocator otherwise, and
respects `-Dforce-gpa` to override the decision. I did this because
I found GeneralPurposeAllocator to be prohibitively slow when doing
multi-threading, even in the context of a debug build.
There is now a second thread pool which is used to spawn each
test case. The stage2 tests are passed the first thread pool. If it were
only multi-threading the stage1 tests then we could use the same thread
pool for everything. However, the problem with this strategy with stage2
is that stage2 wants to spawn tasks and then call wait() on the main
thread. If we use the same thread pool for everything, we get a deadlock
because all the threads end up all hanging at wait() and nothing is
getting done. So we use our second thread pool to simulate a "process pool"
of sorts.
I spent most of the time working on this commit scratching my head trying
to figure out why I was getting ETXTBSY when spawning the test cases.
Turns out it's a fundamental Unix design flaw, already a known, unsolved
issue by Go and Java maintainers:
https://github.com/golang/go/issues/22315https://bugs.openjdk.org/browse/JDK-8068370
With this change, the following command, executed on my laptop, went from
6m24s to 1m44s:
```
stage1/bin/zig build test-cases -fqemu -fwasmtime -Denable-llvm
```
closes#11818
This comment is now deleted because the task is completed in this
commit:
```
// TODO: Update this to behave like `beginComptimePtrLoad` and properly check/use
// `container_ty` and `array_ty`, instead of trusting that the parent decl type
// matches the type used to derive the elem_ptr/field_ptr/etc.
//
// This is needed because the types will not match if the pointer we're mutating
// through is reinterpreting comptime memory.
```
The main strategy is to change the ComptimePtrMutationKit struct so that
instead of `val: *Value` it now returns a tagged union which can be one
of three possibilities:
* The pointer type matches the actual comptime Value so a direct
modification is possible. Before this commit, the implementation
incorrectly assumed this was always the case.
* In the case of needing to write through a reinterpreted pointer, a
mutable base Value pointer is provided along with a byte offset
pointing to the element value in virtual memory.
* Otherwise, it means a compile error must be emitted because one or
both of the types (the owner of the value, or the pointer type being
used to write through) do not have a well-defined memory layout.
After calling beginComptimePtrMutation, the one callsite now switches on
this tagged union and does the appropriate thing. The main new logic is
for the second case, which involves pointer reinterpretation, which now
takes this strategy:
1. write the base value to a memory buffer.
2. perform the pointer store at the proper byte offset, thereby
modifying a subset of the buffer.
3. read the base value from the memory buffer, overwriting the old base
value.
This commit does not change any behavior, but changes the type of
the runtime_index field from u32 to a non-exhaustive enum. This allows
us to put `std.math.maxInt(u32)` only in the enum type definition and
give it an official meaning.
Rather than storing all the shifts in temporaries, we perform the correct
shifting without temporaries. This makes the runtime code more performant
and also the backend code is simplified as we have a singular abstraction.
This does however not support floats of bitsizes
different than 32 or 64. f16, f80, f126 will require
support for compiler-rt and are out-of-scope for this commit.
Signed integers are currently not supported either.
llvm: dump failed module when -femit-llvm-ir set
print_air:
* print fully qualified name
* use Type.fmt and Value.fmtValue, fmtDebug is useless
TypedValue
* handle anon structs and tuples
* fix bugs
This comptime gate is needed to make sure that purely single-threaded
programs don't generate calls to the std.Thread API.
WASI targets successfully build again with this change.
The function `writeDbgInfoNopsBuffered` was based on the function
`pwriteDbgInfoNops`, originally written by me, and then modified to
write to a memory buffer instead of an open file. When writing to a
file, any extra bytes beyond the end of the file extend the size of
the file, and the function body of `pwriteDbgInfoNops` takes advantage
of this when `next_padding_bytes` causes the write to go beyond the
end of the file. However, when writing to a memory buffer, the
underlying array list must be expanded if the write would cause the
buffer to expand.
Note that the current documentation for the `-z noexecstack` is
incorrect. This indicates that an object *does not* require an
executable stack.
This is actually the default of LLD, and there has never been a way to
override this default by passing `-z execstack` to LLD.
This commit removes the redundant `-z noexecstack` option from
zig build-exe/build-lib/build-obj and ignores the option if passed
to zig cc for compatibility.
As far as I can tell, there is no reason for code to require an
executable stack. This option only exists because the stack was
originally executable by default and some programs came to depend
on that behavior. Instead, mprotect(2) may be used to make memory
pages executable.
I'm not really happy with parsing compile errors; I think we should just
be checking that the expected compile error matches the actual rendered
version. I will save that change for a later date however.
And use it to debug a LazySrcLoc in stage2 that is set to a bogus value.
The actual fix in this commit is:
```diff
- try sema.emitBackwardBranch(&child_block, call_src);
+ try sema.emitBackwardBranch(block, call_src);
```
Whenever a `ref` instruction is needed, it is created and saved in
`AstGen.ref_table` instead of being immediately appended to the current
block body. Then, when the referenced instruction is being added to the
parent block (e.g. from setBlockBody), if it has a ref_table entry, then
the ref instruction is added directly after the instruction being referenced.
This makes sure two properties are upheld:
1. All pointers to the same locals return the same address. This is required
to be compliant with the language specification.
2. `ref` instructions will dominate their uses. This is a required property
of ZIR.
A complication arises when a ref instruction refs another ref
instruction. The logic in appendBodyWithFixups must take this into
account, recursively handling ref refs.
Full RELRO is a hardening feature that makes it impossible to perform
certian attacks involving overwriting parts of the Global Offset Table
to invoke arbitrary code.
It requires all symbols to be resolved before execution of the program
starts which may have an impact on startup time. However most if
not all popular Linux distributions enable full RELRO by default for
all binaries and this does not seem to make a noticeable difference
in practice.
"Partial RELRO" is equivalent to `-z relro -z lazy`.
"Full RELRO" is equivalent to `-z relro -z now`.
LLD defaults to `-z relro -z lazy`, which means Zig's current `-z relro`
option has no effect on LLD's behavior.
The changes made by this commit are as follows:
- Document that `-z relro` is the default and add `-z norelro`.
- Pass `-z now` to LLD by default to enable full RELRO by default.
- Add `-z lazy` to disable passing `-z now`.
Since Zig provides @clz and not @ffs (find-first-set), log2 for comptime
integers needs to be computed algorithmically. To avoid hitting the
backward branch quota, this updates log2(x) to use a simple O(log N)
algorithm.
alongside the typical msghdr struct, Zig has added a msghdr_const
type that can be used with sendmsg which allows const data to
be provided. I believe that data pointed to by the iov and control
fields in msghdr are also left unmodified, in which case they can
be marked const as well.
Zig allows multiple extern functions with the same name, and the
backends have to handle this possibility.
For LLVM, we keep a sparse map of collisions, and then resolve them in
flushModule(). This introduces some technical debt that will have to be
resolved when adding incremental compilation support to the LLVM
backend.
* move global into function scope
* clarify comments
* avoid unnecessary usage of std.atomic API
* switch on error instead of `catch unreachable`
* call linux.gettid() instead of going through higher level API and
doing unnecessary casting
* Document deviation from Linux man page, which is identical to musl.
Man page wants always enabled user-provided abort handlers.
Worst case logic bug, which this can introduce:
+ user disables SIGABRT handler to prevent tear down to last safe
state
+ abort() gets called and enables user-provided SIGABRT handler
+ SIGABRT tears down to supposed last safe state instead of crash
+ Application, instead of crashing, continues
* Pid 1 within containers needs special handling.
- fatal signals are not transmitted without privileges,
so use exit as fallback
* Fix some signaling bits
* Add checks in Debug and ReleaseSafe for wrong sigprocmask
This reverts commit 135b91aecd.
"endsWithBreak()" is not a meaningful question to ask and should not be
used this way. A simple example that defeats this logic is:
```zig
export fn entry() void {
outer: {
{
break :outer;
}
return;
}
}
```
Split type relocs into two kinds: local and global. Global relocs
use a global type resolver and calculate offset to the existing
definition of a type abbreviation.
Local relocs use offset in the abbrev section of the containing
atom plus addend to generate a local relocation.
Block statements that end with "break" should not be considered
"noreturn" for the enclosing scope, but other "noreturn" instructions
(return, panic, compile error, etc.) should be. This differentiation
necessitates handling "break" differently from the other "noreturn"
instructions when inside a block statement.
This function took is_ptr: bool and then branched on it three times.
Now, instead, each implementation does no branching and the logic is
easier to follow, both for maintainers and compilers.
I also fixed a bug with TryPtr not ensuring enough capacity in the extra
array.
* Introduce "_ptr" variants of ZIR try instruction to disallow constructs
such as `try` on a pointer value instead of an error union value.
* Disable the "_inline" variants of the ZIR try instruction for now because
we are out of ZIR tags. I will free up some space in an independent commit.
* AstGen: fix tryExpr calling rvalue() on ResultLoc.ref
Implements semantic analysis for the new try/try_inline ZIR
instruction. Adds the new try/try_ptr AIR instructions and implements
them for the LLVM backend.
Fixes not calling rvalue() for tryExpr in AstGen.
This is part of an effort to implement #11772.
gitrev kubkon/zig-yaml 8cf8dc3bb901fac8189f441392fc0989ad14cf71
Calculate line and col info indexed by token index. We can then
re-use this info to track current column number (aka indentation
level) of each "key:value" pair (map) or "- element" (list).
This significantly cleans up the code, and leads naturally to
handling of unindented lists in tbd files.
LLVM optimization passes handle this better, and it allows Zig to
specify pointer parameter attributes such as readonly, nonnull, noalias,
and alignment.
closes#561
This moves some logic from resolveLlvmFunction to updateFunc and takes
advantage of the iteration we already do that takes into account C ABI
lowering, making LLVM parameter attributes accurate for C ABI functions
as well as our own unspecified calling convention.
Related to #11498.
Generally, the load instruction may need to make a copy of an
isByRef=true value, such as in the case of the following code:
```zig
pub fn swap(comptime T: type, a: *T, b: *T) void {
const tmp = a.*;
a.* = b.*;
b.* = tmp;
}
```
However, it only needs to do so if there are any instructions which can
possibly write to memory. When calling functions with isByRef=true
parameters, the AIR code that is generated looks like loads followed
directly by call.
This allows for a peephole optimization when lowering loads: if the load
instruction operates on an isByRef=true type and dies before any side effects
occur, then we can safely lower the load as a no-op that returns its
operand.
This is one out of three changes I intend to make to address #11498.
However I will put these changes in separate branches and merge them
separately so that we can have three independent points on the perf
charts.
After P-256, here comes P-384, also known as secp384r1.
Like P-256, it is required for TLS, and is the current NIST recommendation for key exchange and signatures, for better or for worse.
Like P-256, all the finite field arithmetic has been computed and verified to be correct by fiat-crypto.
This is one out of three changes I intend to make to address #11498.
However I will put these changes in separate branches and merge them
separately so that we can have three independent points on the perf
charts.
This check for primitives is already handled by the generic logic that
checks if the body ends up being empty. I kept this commit in the git
history in case we ever want that nodePrimitive function again in the
future, it might be useful.
The main purpose of this commit is to prepare to implement support for
callconv(), align(), linksection(), and addrspace() annotations on
generic functions where the provided expression depends on comptime
parameters (making the function generic).
It's a rather involved change, so this commit only makes the necessary
changes to AstGen without regressing any behavior, and a follow-up
commit can finish the task by making the enhancements to Sema.
By my quick estimation, the new encoding for functions is a negligible
improvement - along the lines of 0.005% fewer total ZIR bytes on
average. Still, it's nice that this commit, while adding more
data into ZIR, actually ends up reducing the storage size thanks to a
slightly more sophisticated encoding.
Zir.Inst.ExtendedFunc is renamed to Zir.Inst.FuncFancy to eliminate
confusion about it being an extended instruction (it used to be but is
no longer). The encoding for this instruction is completely reworked.
The encoding for Zir.Inst.Func is also changed slightly - when the
return type body length is 1, then only a Zir.Inst.Ref is provided; not
a full body.
linksection() and addrspace() are now communicated via func_fancy ZIR
instruction rather than as part of the corresponding decl. This allows
their expressions to observe comptime parameters.
As demonstrated by this new test case, stage1's functionality is
incorrect since it does not handle slicing from len..len correctly.
stage2 already has the correct behavior here.
all_mask is a value of type sigset_t, which is defined as an array type
[N]u32. However, all_mask references sigset_t.len, but, the array type
does not have a len field. Fix is to use @typeInfo(sigset_t).Array.len
instead.
This tight coupling causes problems for various targets, requires
hacky "get args" functionality, and bungles relative file system paths,
making invalid assumptions about the zig-cache directory.
In short, these are not unit tests; these should be standalone tests
instead.
Reverts e5d4a694ea
Reverts d976456ef6
Reverts dbbda0f41aCloses#11542
Prior to this change, for an example compiler error test case with
multiple identical errors messages such as
```
:1:2: error: foo
:1:2: error: foo
```
the test harness would never increment the error index thus only
marking the very first error message as handled yielding a false
positive.
Additionally, while here, regress `dereference_anyopaque` test case
as not passing on `wasm32-wasi` target.
Add the ability to generate a random, canonical curve25519 scalar,
like we do for p256.
Also leverage the existing CompressedScalar type to represent these
scalars.
SPARC does not have an explicit notion of saving/restoring registers.
The usual windowing mechanism (save/restore/return) already takes care of that
for us.
Additionally:
* Sema: fix array cat/mul not setting the sentinel value
- This required an LLVM backend enhancement to the handling of the
AIR instruction aggregate_init that likely needs to be
propagated to the other backends.
* Sema: report integer overflow of array concatenation in a proper
compile error instead of crashing.
* Sema: fix not using proper pointer address space for array cat/mul
With this change, we are now correctly lowering `sub_with_overflow`
for signed and unsigned integers of register-sized integers (32-
or 64-bit precisely). We also match LLVM's behavior and so, the
condition flags we now set are:
* unsigned:
- `add_with_overflow`: `hs`/`cs` (carry set)
- `sub_with_overflow`: `lo`/`cc` (carry clear)
* signed:
- `add_with_overflow`/`sub_with_overflow`: `vs` (overflow)
readv() is essentially identical to read() except for the buffer type,
this simplifies the API for the caller at the cost of not clearly mapping to the liburing C API.
Reads can be done in two ways with io_uring:
* using a simple buffer
* using a automatic buffer selection which requires the user to have
provided a number of buffers before
ReadBuffer let's the caller choose where the data should be read.
This is needed because pointers to zero-bit types are not necessarily
comptime known, but when doing a load, only the element type having one
possible value is relevant.
* `?E` where E is an error set with only one field now lowers the same
as `bool`.
* Fix implementation of errUnionErrOffset and errUnionPayloadOffset to
properly compute the offset of each field. Also name them the same
as the corresponding LLVM functions and have the same function
signature, to avoid confusion. This fixes a bug where wasm was
passing the error union type instead of the payload type.
* Fix C backend handling of optionals with zero-bit payload types.
* C backend: separate out airOptionalPayload and airOptionalPayloadPtr
which reduces branching and cleans up control flow.
* Make Type.isNoReturn return true for error sets with no fields.
* Make `?error{}` have only one possible value (null).
Based on the size of the payload the native backends will lower
the error union with its fields (errorset & payload) in the correct order.
e.g. ErrorA!u8 will first lower the error set's value and then the payload.
In the event of ErrorA!u32 will lower the payload first.
* Sema: avoid unnecessary safety checks when an error set is empty.
* Sema: make zirErrorToInt handle comptime errors that are represented
as integers.
* Sema: make empty error sets properly integrate with
typeHasOnePossibleValue.
* Type: correct the ABI alignment and size of error unions which have
both zero-bit error set and zero-bit payload. The previous code did
not account for the fact that we still need to store a bit for
whether there is an error.
* LLVM: lower error unions possibly with the payload first or with the
error code first, depending on alignment. Previously it always put
the error code first and used a padding array.
* LLVM: lower functions which have an empty error set as the return
type the same as anyerror, so that they can be used where
fn()anyerror function pointers are expected. In such functions, Zig
will lower ret to returning zero instead of void.
As a result, one more behavior test is passing.
This is a temporary addition to stage2 in order to match stage1 behavior,
however the end-game once the lang spec is settled will be to use a global
InternPool for comptime memoized objects, making this behavior consistent
across all types, not only string literals. Or, we might decide to not
guarantee string literals to have equal comptime pointers, in which case
this commit can be reverted.
Prior to this change we would assume the ABI for Apple targets to
be GNU which could result in subtle errors in LLVM emitting calls
to non-existent system libc provided functions such as `_sincosf`
which is a GNU extension and as such is not provided by macOS for example.
This would result in linker errors where the linker would not be
able to find the said symbol in `libSystem.tbd`.
With this change, we now correctly identify macOS (and other Apple
platforms) as having ABI `unknown` which translates to unspecified
in LLVM under-the-hood:
```
// main.ll
target triple = "aarch64-unknown-macos-unknown"
```
Note however that we never suffix the target OS with target version
such as `macos11` or `macos12` which means we fail to instruct LLVM
of potential optimisations provided by the OS such as the availability
of function `___sincosf_stret`. I suggest we investigate that in a
follow-up commit.
MCValue.cpsr_flags replaces
MCValue.compare_flags_{signed,unsigned}. This simplifies a lot of
stuff and enables an MCValue to represent only the overflow bits in
the CPU (previously, it was only possible to represent a register +
the overflow bits).
The previous commit caused LLVM module verification failure because we
attemped to bitcast LLVM pointers to i64 parameters. This is exactly
what we want, however it's technically not allowed according to LLVM's
type system. It could have been fixed trivially by using ptrtoint
instead of bitcast in the case of pointers, however, out of concern for
inttoptr being problematic for the optimizer, I put in special code to
detect when a given parameter can be treated as its actual type rather
than an integer type. This makes Zig's output LLVM IR closer to what
Clang outputs.
The previous implementation of calling conventions was hacky and broken.
This commit reworks lowerFnParamTy into iterateParamTypes which returns
enum tags indicating how to handle each parameter. This is then used in
the three places that matter:
* lowering a function type to llvm type
* converting function parameters to the canonical type representation
(with respect to isByRef).
* converting canonical type representation to function arguments at
callsites (again with respect to isByRef).
As a result, we are one step closer to the C ABI tests passing. Before
this commit, attempting to build them crashed the compiler. I isolated
the broken function and verified that it now is lowered correctly. I
will keep working on this one piece at a time until all the C ABI tests
pass, and then I will enable all of them in the CI.
Motivation: the behavior test that is now passing.
The main change in this commit is introducing `Type.abiSizeAdvanced`,
`Value.Tag.lazy_size`, and adjusting `Sema.zirSizeOf` to take advantage
of these.
However, the bulk of lines changed in this commit ended up being moving
logic from value.zig and type.zig into Sema.zig. This logic had no
business being in Type/Value as it was only called from a Sema context,
and we need access to the Sema context for error reporting when a lazy
Value is resolved.
Also worth mentioning is that I bumped up the comptime `@floatToInt`
implementation from using f64 to f128.
`@call` allows specifying the modifier explicitly, however it can still
appear in a context that overrides the modifier. This commit adds flags
to the BuiltinCall ZIR encoding. Since we have unused bits I also threw
in the ensure_result_used mechanism.
I also deleted a behavior test that was checking for bound function
behavior where I think stage2 behavior is correct and stage1 behavior
is incorrect.
Warnings about non-implemented `-z nocopyreloc` are common when
compiling go code (including Go's tests themselves). Let's just
make it stop complaining.
This way, we do not have to tweak the `RegisterManager` to handle
multiple register types - we have one linear space instead. Additionally
we can use the bitset itself to separate the registers into overlapping
(the ones that are aliases of differing bitwidths) and nonoverlapping
classes (for example, AVX registers do not overlap general purpose
registers, thus they can be allocated simultaneously).
Another huge benefit of this simple approach is the fact that we can
still refer to *all* registers regardless of their class via enum
literals which makes the code so much more readable.
Finally, `RegisterLock` is universal across different register classes.
This test was also covering this behavior:
```zig
test "equality of pointers to comptime const" {
const a: i32 = undefined;
comptime assert(&a == &a);
}
```
This check belongs in its own behavior test which isolates this
behavior; not bundled along with a C pointer test.
This also fixes the instruction for all other integer bitsizes,
as it was previously assuming to always be a bool.
128 bit substraction was also fixed as it contained a bug where it swapped
lhs with rhs.
This also implments wrapping for arbitrary integer widths between 64 and 128.
`@truncate` was fixed where the wasm types between operand and result differentiated.
We solved this by first casting and then wrapping.
`airMaxMin` was slightly updated to automatically support 128 bit integers,
by using the `cmp` function, instead of doing it manually. This makes the function
more maintanable as well.
`ctz` and `clz` now support 128 bit integers, while updating the previous implementation
also.
We now pass the correct wasm type when the return type is a 128-bit integer.
When a function accepts a 128-bit integer, we now allocate space on the virtual stack
and store both arguments within that space as currently all following instructions
assume the 128 bit integer doesn't live in a local, but the stack.
This implements support for all compare operations on a 128bit integer,
for both signed and unsigned integers.
The new implementation is almost more efficient as it requires no control-flow,
unlike the old implementation which used a block with breaks.
Previously, updating the `SYS` enum for each architecture required
manually looking at the syscall tables and inserting any new additions.
This commit adds a tool, `generate_linux_syscalls.zig`, that automates
this process using the syscall tables in the Linux source tree. On
architectures without a table, it runs `zig cc` as a pre-processor to
extract the system-call numbers from the Linux headers.
This avoids the following error:
```
error: incompatible pointer types passing 'int64_t *' (aka 'long long *') to parameter of type 'long *'
overflow = __builtin_saddl_overflow(lhs, rhs, res);
^~~
```
My previous understanding was that this error would not occur because
prior to this line we check that int64_t is equivalent to long, like
this:
```c
```
However, it appears that this is still a warning in C if int64_t is
primarily aliased to `long long`, even though `long` and `long long` are
the same thing.
The implementation for add_with_overflow and sub_with_overflow is now a lot
more robust and takes account for signed integers and arbitrary integer bitsizes.
The final output is equal to that of the LLVM backend.
Most of the work here was additions to zig.h. The lowering code is
mainly responsible for calling the correct function name depending on
the operand type.
Some of the compiler-rt calls here are not implemented yet and are
non-standard symbols due to the C programming language not needing them.
After this commit, the behavior tests with -ofmt=c are passing again.
Originally I thought interleaving AIR with MIR will be useful, however
as it stands, I have used it very sporadically, and recently, not at
all, and I do not think anyone else is actually using it. If there is
a simple error such as a wrong instruction emitted,
`objdump` is perfectly capable of narrowing it down, while if there's
something more subtle happening, regardless of having `--verbose-mir`
functionality or not, you still gotta go via the debugger which
offers a better view at interleaved source program with the emitted
machine code. Finally, I believe `-femit-asm` when we add it will offer a
more generic substitute.
While calling `next` an error can occur while parsing the file.
However, we don't set the filename that is currently being processed, until `next` completed successfully.
This means that for invalid test names, the wrong filename was being displayed in the panic message.
The fix is to retrieve the correct filename when an error occurs and then setting the filename appropriately.
This matches master branch. We can look into adding more target coverage
as we switch to stage2. As it stands, this works around having to
duplicate the "Executor" logic to figure out when to not run the tests
due to them being non-native.
* migrate runtime safety tests to the new test harness
- this required adding compare output / execution support for stage1
to the test harness.
* rename `zig build test-stage2` to `zig build test-cases` since it now
does quite a bit of stage1 testing actually. I named it this way
since the main directory in the source tree associated with these
tests is "test/cases/".
* add some documentation for the test manifest format.
Rename all references of sparcv9 to sparc64, to make Zig align more with
other projects. Also, added new function to convert glibc arch name to Zig
arch name, since it refers to the architecture as sparcv9.
This is based on the suggestion by @kubkon in PR 11847.
(https://github.com/ziglang/zig/pull/11487#pullrequestreview-963761757)
the list parameter should be a multi-item pointer rather than a single-item
pointer. see: https://linux.die.net/man/2/setgroups
> setgroups() sets the supplementary group IDs for the calling process...
> the size argument specifies the number of supplementary group IDs in the buffer pointed to by list.
The "slicing operator with sentinel" runtime safety test cases should
all have been compile errors in their current forms. In this commit I
adjust them to use runtime-known slices before triggering the runtime
safety.
Furthermore the test cases did not actually have unique names.
- Test: Fix bucket counting. Previously, the first hit was not counted.
This off-by-one error slightly increased the mean of `*_total_variance`,
which decreased the acceptance rate for a particular random seed
from 95% to 92.6%. (Irrelevant for test failure because the seed is fixed.)
- Improve comments
EnvMap provides the same API as the previously used BufMap (besides `putMove` and `getPtr`), so usage sites of `getEnvMap` can usually remain unchanged.
For non-Windows, EnvMap is a wrapper around BufMap. On Windows, it uses a new EnvMapWindows to handle some Windows-specific behavior:
- Lookups use Unicode-aware case insensitivity (but `get` cannot return an error because EnvMapWindows has an internal buffer to use for lookup conversions)
- Canonical names are returned when iterating the EnvMap
Fixes#10561, closes#4603
When handling the `negate` ZIR instruction, Zig now checks for a
comptime operand and handles it as a special case rather than lowering
it as `0 - x` so that the expression `-x` where `x` is a floating point
value known at compile-time, will get the negative zero bitwise
representation.
* back out the changes to RunStep
* move the disabled test to the .cpp code and avoid a confusing
name-collision with the _LIBCPP macro prefix
* fix merge conflict with the edits to the same test that ensure global
initializers are called.
Now this branch is only concerned with single-threaded targets and
passing the correct macro defines to libc++.
```
$ valgrind ./zig test ../test/behavior.zig -target powerpc-linux-musl -lc -I../test
==2828778== Invalid read of size 1
==2828778== at 0x6EA0265: LLVMSetVisibility (in /home/andy/Downloads/zig/build/zig)
==2828778== by 0x1BCE60B: do_code_gen(CodeGen*) (codegen.cpp:9031)
==2828778== by 0x1BD51E2: codegen_build_object(CodeGen*) (codegen.cpp:10610)
==2828778== by 0x1BA5C17: zig_stage1_build_object (stage1.cpp:132)
==2828778== by 0xE61E24: Module.build_object (stage1.zig:149)
==2828778== by 0xC3D4CE: Compilation.updateStage1Module (Compilation.zig:5025)
==2828778== by 0xC3117E: Compilation.performAllTheWork (Compilation.zig:2691)
==2828778== by 0xC2A3ED: Compilation.update (Compilation.zig:2098)
==2828778== by 0xBB9D1F: main.updateModule (main.zig:3104)
==2828778== by 0xB16B75: main.buildOutputType (main.zig:2793)
==2828778== by 0xAD0526: main.mainArgs (main.zig:225)
==2828778== by 0xACFCB9: main (stage1.zig:48)
```
Since the plan is to ship stage3 for Zig 0.10.0, the stage1
implementation of this hardly matters.
* Rename std.builtin.GlobalVisibility to std.builtin.SymbolVisibility
* Add missing compile error. From the LLVM language reference: "A
symbol with internal or private linkage must have default
visibility."
This commit adds the ability to emit the following debug sections:
.debug_info
.debug_abbrev
.debug_line
.debug_str
Line information and files are now being loaded correctly by browser debuggers.
This implements parts to commit a decl's debug information into
a linear memory buffer. The goal is to write this buffer at once
after we finished linking.
Maps lines and columns between wasm bytecode and Zig source code.
While this supports prologue and epilogue information, we need to add
support for performing relocations as the offsets are relative to the code section,
which means we must relocate it according to the atom offset's offset while keeping function count
in mind as well (due to leb128 encoding).
Implements very basic debug information for locals.
For now it only implements debug info when the variable is stored within a
Wasm local. The goal is to support those that live in the data section (virtual stack).
When exporting math functions for Windows, we provide weak exports of
'l' variants rather than weak aliases. We still use aliases on other
operating systems so that the 'l' variants have one less jump
instruction in this case.
As we now store negative signed integers as two's complement,
we must also ensure that when truncating a float, its value is wrapped
around the integer's size.
This also splits `@mulWithOverflow` into its own function to make
the code more maintainable and reduce branching.
stage2: change logic for detecting whether the main package is inside
the std package. Previously it relied on realpath() which is not portable.
This uses resolve() which is how imports already work.
* stage2: fix cleanup bug when creating Module
* flatten lib/std/special/* to lib/*
- this was motivated by making main_pkg_is_inside_std false for
compiler_rt & friends.
* rename "mini libc" to "universal libc"
This improves the ABI alignment resolution code.
This commit fully enables the MachO linker code in stage3. Note,
however, that there are still miscompilations in stage3.
This prevents a nasty type of bugs where we accidentally unfreeze
a register that was frozen purposely in the outer scope, risking
accidental realloc of a taken register.
Fix CF flags spilling on aarch64 backend.
When a signed integer is negative, the integer will be stored as a two's complement,
rather than its signed value. Instead, we verify the signed bits during arithmetic operations.
This fixes signed cases of `@mulWithOverflow`.
If the hw doesn't have support for exotic floating-point types such
as `f80`, we lower the call to a compiler-rt function call instead.
I've added a behavior test specifically targeting this use case which
now passes on `aarch64-macos`.
Additionally, this commit makes it possible to successfully build
stage3 on `aarch64-macos`. We can print the compiler's help message,
however, building with it needs a little bit more love still.
We can't yet run the behavior tests with stage3, but at least we can run
them with stage2, and we can use the proper test matrix.
This commit also adds use_llvm and ofmt to the zig build system.
* sret logic needed a check for hasRuntimeBits()
* lower f128 on windows targets with the "sse" class rather than
"memory". For reference, clang emits a compile error when __float128
is used with the MSVC ABI, saying that this type is not supported.
The docs for the x64 calling convention have both of these sentences:
- "Any argument that doesn't fit in 8 bytes, or isn't 1, 2, 4, or 8 bytes,
must be passed by reference."
- "All floating point operations are done using the 16 XMM registers."
* For i128, however, it is clear that the Windows calling convention
wants such an object to be passed by reference. I fixed the LLVM
lowering for function parameters to make this work.
So far it's supported by the LLVM backend only. I recommend for the
other backends to wait for the resolution of #10761 before adding
support for this feature.
Ideally on Windows, static libraries look like "foo.lib". However, CMake
and other build systems will unfortunately produce static libraries that
instead look like "libfoo.a". This patch makes Zig's CLI resolve "-lfoo"
arguments into static libraries that match this other pattern.
This patch fixes an issue with zig-bootstrap where it won't find the
LLVM, Clang, and LLD libraries.
These targets now have a similar disagreement with LLVM about the
alignment of 128-bit integers as x86_64:
* riscv64
* powerpc64
* powerpc64le
* mips64
* mips64el
* sparcv9
See #2987
For x86_64, LLVMABIAlignmentOfType(i128) reports 8. However I think 16
is a better number for two reasons:
1. Better machine code when loading into SIMD register.
2. The C ABI wants 16 for extern structs.
ZIR instructions updated: atomic_load, atomic_rmw, atomic_store, cmpxchg
These no longer construct a pointer type as the result location. This
solves a TODO that was preventing the pointer from possibly being
volatile, as well as properly handling allowzero and addrspace.
It also allows the pointer to be over-aligned, which may be needed
depending on the target. As a consequence, the element type needs to be
communicated in the ZIR. This is done by strategically making one of the
operands be ResultLoc.ty instead of ResultLoc.coerced_ty if possible, or
otherwise explicitly adding elem_type into the ZIR encoding, such as in
the case of atomic_load.
The pointer type of atomic operations is now checked in Sema by coercing
it to an expected pointer type, that maybe over-aligned according to
target requirements.
Together with the previous commit, Zig now has smaller alignment for
large integers, depending on the target, and yet still has type safety
for atomic operations that specially require higher alignment.
Prior to this commit, the logic for ABI size and ABI alignment for
integers was naive and incorrect. This results in wasted hardware as
well as undefined behavior in the LLVM backend when we memset an
incorrect number of bytes to 0xaa due to disagreeing with LLVM about the
ABI size of integers.
This commit introduces a "max int align" value which is different per
Target. This value is used to derive the ABI size and alignment of all
integers.
This commit makes an interesting change from stage1, which treats
128-bit integers as 16-bytes aligned for x86_64-linux. stage1 is
incorrect. The maximum integer alignment on this system is only 8 bytes.
This change breaks the behavior test called "128-bit cmpxchg" because on
that target, 128-bit cmpxchg does require a 16-bytes aligned pointer to
a 128 bit integer. However, this alignment property does not belong on
*all* 128 bit integers - only on the pointer type in the `@cmpxchg`
builtin function prototype. The user can then use an alignment override
annotation on a 128-bit integer variable or struct field to obtain such
a pointer.
With this improved iterator, type of test is now inferred from
the filename, enabling us to put all cases in one common parent
directory, and iterate over that, thus automating a lot of tasks.
Previously, TrailerFlags was unreferenced in std,
so its tests were never run.
Also, fix the use of `default_value` whose type was changed in
f4a249325e (#10766).
Adding `unreachable` prevents the futex code from being inspected during
a single-threaded build. Without futex, first draft BYOS packages don't
need to implement `nanosleep` to get a single-threaded "hello world"
program working.
Use of `assert()` did not achieve the desired effect of avoiding futex
in a single-threaded build.
This was a bit trickier than it should be due to symbol conflicts with
zig's compiler-rt implementation. We attempt to use weak linkage in
our compiler-rt, but this does not seem to be working in all cases. I
manually disabled export of the problematic compiler-rt math functions
in order to cross compile musl's libc.so for all targets as input to
`tools/gen_stubs.zig`.
Other than that, this update went fairly smoothly. Quite a few
additional symbols were added to the blacklist in `tools/gen_stubs.zig`
due to recent reorganization of zig's compiler-rt.
* outputs can have names and be referenced with template replacements
the same as inputs.
* fix print_air.zig not decoding correctly.
* LLVM backend: use a table for template names for simplicity
The previous float-parsing method was lacking in a lot of areas. This
commit introduces a state-of-the art implementation that is both
accurate and fast to std.
Code is derived from working repo https://github.com/tiehuis/zig-parsefloat.
This includes more test-cases and performance numbers that are present
in this commit.
* Accuracy
The primary testing regime has been using test-data found at
https://github.com/tiehuis/parse-number-fxx-test-data. This is a fork of
upstream with support for f128 test-cases added. This data has been
verified against other independent implementations and represents
accurate round-to-even IEEE-754 floating point semantics.
* Performance
Compared to the existing parseFloat implementation there is ~5-10x
performance improvement using the above corpus. (f128 parsing excluded
in below measurements).
** Old
$ time ./test_all_fxx_data
3520298/5296694 succeeded (1776396 fail)
________________________________________________________
Executed in 28.68 secs fish external
usr time 28.48 secs 0.00 micros 28.48 secs
sys time 0.08 secs 694.00 micros 0.08 secs
** This Implementation
$ time ./test_all_fxx_data
5296693/5296694 succeeded (1 fail)
________________________________________________________
Executed in 4.54 secs fish external
usr time 4.37 secs 515.00 micros 4.37 secs
sys time 0.10 secs 171.00 micros 0.10 secs
Further performance numbers can be seen using the
https://github.com/tiehuis/simple_fastfloat_benchmark/ repository, which
compares against some other well-known string-to-float conversion
functions. A breakdown can be found here:
0d9f020f1a/PERFORMANCE.md (commit-b15406a0d2e18b50a4b62fceb5a6a3bb60ca5706)
In summary, we are within 20% of the C++ reference implementation and
have about ~600-700MB/s throughput on a Intel I5-6500 3.5Ghz.
* F128 Support
Finally, f128 is now completely supported with full accuracy. This does
use a slower path which is possible to improve in future.
* Behavioural Changes
There are a few behavioural changes to note.
- `parseHexFloat` is now redundant and these are now supported directly
in `parseFloat`.
- We implement round-to-even in all parsing routines. This is as
specified by IEEE-754. Previous code used different rounding
mechanisms (standard was round-to-zero, hex-parsing looked to use
round-up) so there may be subtle differences.
Closes#2207.
Fixes#11169.
* edwards25519: fix X coordinate of the base point
Reported by @OfekShochat -- Thanks!
* edwards25519: reduce public scalar when the top bit is set, not cleared
This is an optimization for the unexpected case of a scalar
larger than the field size.
Fixes#11563
* edwards25519: add a test implicit reduction of invalid scalars
Instead of doing heterogeneous comparison at comptime. This makes the
following test pass (as it already does for stage1):
```zig
test {
const x: f64 = 12.34;
expect(x == 12.34);
}
```
There is already behavior test coverage for this, however, other bugs in
`std.fmt.parseFloat` are masking the failures.
From a language specification perspective, this makes sense because it
makes comptime comparisons with comptime_float work the same way they
work with runtime comparisons.
Maybe after we have incremental compilation metadata serialization
and non-LLVM backends, it will make sense to switch this back. For now,
however, this makes successive `zig build` commands much faster.
Just like for Struct in 8238d4b335, in the
case of ErrorUnion struct we need to return a compound literal "(T){...}"
instead of just "{}", which is invalid code when used in e.g. a "return"
expression.
* Sema: Correctly determine whether array_cat lhs and rhs are single ptrs
Many-pointers are also not single-pointers and wouldn't be considered
here. This commit makes the conditions use the appropriately-named
isSinglePointer instead.
* Sema: Correctly obtain ArrayInfo for many-pointer concatenation
Many-pointers at comptime have a known size like slices and can be used
in array concatenation. This fixes a stage1 regression.
* test: Add comptime manyptr concatenation test
Co-authored-by: sin-ack <sin-ack@users.noreply.github.com>
Instead, just return ChildProcess directly. This structure does not
require a stable address, so we can put it on the stack just fine. If
someone wants it on the heap they should do.
const proc = try allocator.create(ChildProcess);
proc.* = ChildProcess.init(args, allocator);
18d6523888 regressed compiler-rt tests for
stage1 because it removed a workaround. I updated the comment to better
explain what exactly the workaround is so that it won't happen again.
This reverts commit 75c9936737, reversing
changes made to 7f13f5cd5f.
I don't think `runZigBuild` belongs in std.testing. We already have
`test/standalone/*` for this.
Additionally test names should explain what they are testing rather than
referencing GitHub issue numbers.
This reverts commit 7f13f5cd5f.
I'd like to review this one before it goes in. This is an awfully
specific API that I don't think belongs in std.testing. Also I don't
want any code snippets in doc strings. We have doctests for that.
To correctly handle multiple backends crossed with multiple targets,
we need to push all elements in separate allocated arrays rather
than operate on raw iterators. Hence, introduce `getConfigForKeyAlloc`.
Provide default parsers for obvious config options such as
`CrossTarget` or `Backend` (or any enum for that matter).
Unroll iterator loops into multiple cases - we need to create
a Cartesian product for all possibilities specified in the
test manifest.
* remove need for manual string concatenation for building binaries in test blocks
* include small program snippet to show how to get binary path with subslicing
Before this change, struct {f80, f80} targeting i386-windows-msvc lowers
to
```llvm
%"std.testing.struct:78:61.6" = type { x86_fp80, [6 x i8], x86_fp80, [6 x i8] }
```
which has an incorrect ABI size of 40. After this change, the struct
lowers to
```llvm
%"std.testing.struct:78:61.6" = type { x86_fp80, [4 x i8], x86_fp80, [4 x i8] }
```
which has the correct ABI size of 32, and properly aligns the second
field to 16 bytes.
The other place that calculates field padding (lowering of constant
values in codegen.cpp) already correctly calls LLVMABISizeOfType
rather than LLVMStoreSizeOfType.
This fixes the compiler-rt tests for i386-windows in this branch.
These are only as accurate as f64 even for f128 comptime functions. This
is OK for now; improvements will come with the launch of self-hosted
(#89) and enhancements to compiler-rt implementations.
This is to account for the small differences in math functions of
different libcs. For example, if the compiler links against glibc,
but the target is musl libc, then these values might be
slightly different.
Arguably, this is a bug in the compiler because comptime should
emulate the target, including rounding errors in libc math
functions. However that behavior is not what this particular test
is intended to cover.
The reason for having `@tan` is that we already have `@sin` and `@cos`
because some targets have machine code instructions for them, but in the
case that the implementation needs to go into compiler-rt, sin, cos, and
tan all share a common dependency which includes a table of data. To
avoid duplicating this table of data, we promote tan to become a builtin
alongside sin and cos.
ZIR: The tag enum is at capacity so this commit moves
`field_call_bind_named` to be `extended`. I measured this as one of
the least used tags in the zig codebase.
Fix libc math suffix for `f32` being wrong in both stage1 and stage2.
stage1: add missing libc prefix for float functions.
* std.math.snan: fix compilation error. Also make it and nan inline.
* LLVM: use a proper enum type for float op instead of enum literal.
Also various cleanups.
* LLVM: use LLVMBuildVectorSplat for vector splat AIR instruction.
- also the bindings had parameter order wrong
* LLVM: additionally handle f16 lowering. For now all targets report OK
but I think we will need to add some exceptions to this list.
Updates stage2 to manually lower softfloat operations for all unary
floating point operations and arithmetic.
Softfloat support still needs to be added for conversion operators
(float<->float and int<->float)
* unify the logic for exporting math functions from compiler-rt,
with the appropriate suffixes and prefixes.
- add all missing f128 and f80 exports. Functions with missing
implementations call other functions and have TODO comments.
- also add f16 functions
* move math functions from freestanding libc to compiler-rt (#7265)
* enable all the f128 and f80 code in the stage2 compiler and behavior
tests (#11161).
* update std lib to use builtins rather than `std.math`.
This implements the C-ABI convention as specified by:
https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md
While not an official specification, it's the ABI that is output by clang/LLVM.
As we use LLVM to compile compiler-rt, and want to integrate with C-libraries,
we follow the same convention when the calling convention results in 'C'.
This function is codegen'd incorrectly in stage2, since it fails to
generate the correct soft-float operations. This will be fixed once
issue #11161 is implemented
This reverts commit a430630002.
Wait a minute, I'm sorry, I need to revert this. The whole premise
of this change is broken because the point of the hash is that it tells
whether the same compilation has been done before. This requires items
to be added to the hash in the same sequence every time. This means that
introducing a lock is fundamentally broken because the order needs to be
the same in future runs of the compiler, and not decided by threads
racing against each other.
The proper solution to this is to, in whole cache mode, append the hash
inputs to some data structure, and then after the compilation is
complete, do some kind of sorting on the hash inputs so that they will
be the same order every time, then apply them in sequence. No lock on
the Cache object is needed for this scheme.
Split big test into the two separate things it is testing.
Add missing checks to the test which revealed the test is not actually
passing yet for the C backend.
For parameters and return types of functions with the C calling
convention, the LLVM backend now has a special lowering for the function
type that makes the function adhere to the C ABI. The AIR instruction
lowerings for call, ret, and ret_load are adjusted to bitcast the real
type to the ABI type if necessary.
More work on this will need to be done, however, this improvement is
enough that stage3 now passes all the same behavior tests that stage2
passes - notably, translate-c no longer has a segfault due to C ABI
issues with Zig's Clang C API wrapper.
This makes stage2 and stage3 have different cache namespaces, so that
building something with stage3 does not try to reuse the same cached
artifacts as were produced by stage2. This makes sense since the code
of stage3 is produced by the self-hosted compiler, whereas the code of
stage2 is produced by the bootstrap compiler. Note also that stage4 and
stage3 will share the same zig_backend, end hence cache namespace.
Ideally stage4 and stage3 are identical binaries, so this checks out.
For those souls looking for a zig `size_t` equivalent, and not
lucky/educated enough (that was me yesterday) to know it's the same as
`uintptr_t`.
From a recent discussion on IRC.
With this change, it is now possible to safely call
`var di = std.debug.openSelfDebugInfo(gpa)`. Calling then
`di.deinit()` on the object will correctly free all allocated
resources.
Ensure we store the result of `mmap` with correct alignment.
Rather than allocating Decl objects with an Allocator, we instead allocate
them with a SegmentedList. This provides four advantages:
* Stable memory so that one thread can access a Decl object while another
thread allocates additional Decl objects from this list.
* It allows us to use u32 indexes to reference Decl objects rather than
pointers, saving memory in Type, Value, and dependency sets.
* Using integers to reference Decl objects rather than pointers makes
serialization trivial.
* It provides a unique integer to be used for anonymous symbol names,
avoiding multi-threaded contention on an atomic counter.
With this change, it is now possible to safely call
`var di = std.debug.openSelfDebugInfo(gpa)`. Calling then
`di.deinit()` on the object will correctly free all allocated
resources.
While this code probably could do with some love and a redesign,
this commit fixes the allocations by making sure we explicitly
pass an allocator where required, and we use arenas for temporary
or narrowly-scoped objects such as a `Die` (for `Die` in particular,
not every `FormValue` will be allocated - we could duplicate, or
we can use an arena which is the proposal of this commit).
* Remove the Allocator field; instead it must be passed in as a
parameter to any function that needs it.
* Rename `push` to `append` and `pushMany` to `appendSlice` to match
the conventions set by ArrayList.
When a child process with stdin, stdout behavior set to pipe is
ran on macos it used to hang which has been fixed. Issue existed because
we forgot to call `posix_spawn_file_actions_addclose` syscall on user
exposed file descriptor which resulted on file descriptor not closing
properly.
So that people can start experimenting with compiling their projects
with the self-hosted compiler.
I expect this commit to be reverted after #89 is closed.
When the last instruction is a debug instruction, the type of it is void.
Similarly for 'noreturn' emit an 'unreachable' instruction to tell the wasm-validator
the path cannot be reached.
Also respect the '--strip' flag in the self-hosted wasm linker and not emit a 'name' section
when the flag is set to `true`.
There were a few minor bugs in the rounding behavior and Inf/NaN
handling for the f80 __addxf3 and __subtf3 functions.
This change updates the original generic implementation to correctly
handle f80 floats, including the explicit integer bit.
This change adds support for locating the Zig executable and the library
and global cache directories, based on looking in the fixed "/zig" and
"/cache" directories.
Since our argv[0] on WASI is just the basename (any absolute/relative
path information is deleted by the runtime), there's very limited
introspection we can do on WASI, so we rely on these fixed directories.
These can be provided on the command-line using `--mapdir`, as follows:
```
wasmtime --mapdir=/cwd::. --mapdir=/cache::"$HOME/.cache/zig" --mapdir=/zig::./zig-out/ ./zig-out/bin/zig.wasm
```
Two major changes here:
1. We store the CWD as a simple `[]const u8` and lookup Preopens for
every absolute or CWD-referenced file operation, based on the
Preopen with the longest match (i.e. most specific path)
2. Preorders are normalized to POSIX absolute paths at init time.
Behavior depends on the "cwd_root" parameter of `initPreopensWasi`:
`cwd_root` is used for any Preopens that start with "."
For example:
"./foo/bar" - inits to -> "{cwd_root}/foo/bar"
"foo/bar" - inits to -> "/foo/bar"
"/foo/bar" - inits to -> "/foo/bar"
`cwd_root` must be an absolute path.
Using "/" as `cwd_root` gives behavior similar to wasi-libc.
According to Apple docs, the long double type is a double precision
IEEE754 binary floating-point type, which makes it identical to the
double type. This behavior contrasts to the standard specification,
in which a long double is a quad-precision, IEEE754 binary,
floating-point type.
Thus, we need to take this into account when using the compiler
intrinsics so that we select the correct function version for
FloatMulAdd.
The call to `makeDir` for the top-level component of `sub_path`
can return `error.FileNotFound` if the directory represented by
`self` has been deleted.
Fixes#11397
Some SPARC CPUs (particularly old and/or embedded ones) only has atomic
TAS instruction available (`ldstub`). This adds support for emitting
that instruction in the spinlock.
Previously, the data segments were being aligned twice.
This caused us to overalign the segment and therefore allocate a much larger
size for each segment than was required. This fix ensures we align and set the size
just once, ensuring semantically correct binaries as well as smaller binaries.
When linking with an object file, verify if a relocation is a table index relocation.
If that's the case, add the relocation target to the function table.
This fixes a memory leak when an object file contains one or more element sections which
then contains one or more function indexes. This commit ensures the slice of index functions
for each element section will be freed upon resource deallocation also.
* The `@bitCast` workaround is removed in favor of `@ptrCast` properly
doing element casting for slice element types. This required an
enhancement both to stage1 and stage2.
* stage1 incorrectly accepts `.{}` instead of `{}`. stage2 code that
abused this is fixed.
* Make some parameters comptime to support functions in switch
expressions (as opposed to making them function pointers).
* Avoid relying on local temporaries being mutable.
* Workarounds for when stage1 and stage2 disagree on function pointer
types.
* Workaround recursive formatting bug with a `@panic("TODO")`.
* Remove unreachable `else` prongs for some inferred error sets.
All in effort towards #89.
Without this, it may happen we write the globals without extending
the symtab section header's size. This can potentially lead to
clobbering some data in the file, or simply omitting the globals
from the symtab when displaying with support tooling such as `readelf`.
The problem was that types of non-anytype parameters were being included
as part of the check to see if generic function instantiations were
equal. Now, Module.Fn additionally stores the information for whether each
parameter is anytype or not. `generic_poison` cannot be used to signal
this because the type is still needed for comptime arguments; in such
case the type will not be present in the newly generated function
prototype.
This presented one additional challenge: we need to compare equality of
two values where one of them is post-coercion and the other is not. So
we make some minor adjustments to `Type.eql` to support this. I think
this small complexity tradeoff is worth it because it means the compiler
does much less work on the hot path that a generic function is called
and there is already an existing matching instantiation.
closes#11146
Fix to call siftDown on the removed index instead of always on index 0.
Updated test to a test that fails before and passes now.
PriorityDequeue does not have this issue.
Looks like d3f87f8ac0 fixed the standard cases of dir renaming, but the edge cases (renaming onto an existing empty/non-empty directory) are still behaving differently than on non-Windows.
Sometimes we will want to generate debug info for a constant that
has been lowered to memory and not copied anywhere else. For this
we will need to defer resolution on PIE platforms until all locals
(including GOT entries) have been allocated.
Prior to this, Liveness encoded `asm`, `call`, and `aggregate_init` with
a single 32-bit integer, allowing up to 35 operands (3 are provided by
the regular tomb_bits). However, the Zig language allows function calls
with more than 35 arguments, inline assembly with more than 35 inputs,
and anonymous tuples with more than 35 elements.
The new encoding stores an index to the extra array instead of the bits
directly, and then as many extra elements as needed to encode all the
operands. The MSB is used as a flag to tell which element is the last
one, allowing for 31 bits per element.
Prior to this, print_air did not bother correctly printing tombstones
for these instructions; now it does.
In addition to updating the BigTomb iteration logic in the machine code
backends, this commit extracts the common logic into the Liveness namespace.
These defines are present on some FreeBSD systems, regardless of
whether the system is big- or little- endian. This was causing the
FreeBSD CI to be incorrectly flagged as big-endian.
Resolves https://github.com/ziglang/zig/issues/11391
Also split the Dir.rename on directories test into 3 tests:
- General rename of a directory
- Rename of a directory onto an existing empty directory
- Rename of a directory onto an existing non-empty directory
The only new case is the rename onto an existing empty directory, but splitting the tests this way made them much more understandable.
Currently transitive system library dependencies are always linked using
linkSystemLibrary() and therefore pkg-config even if they were
originally specified with linkSystemLibraryName() instead. This causes
problems in practice for projects needing total control over exactly
what library is linked, such as the mach game engine.
This is fixed by keeping track of whether libraries are to be linked
with pkg-config or not and holding off on actually running pkg-config
until after transitive dependency resolution in LibExeObjStep.make().
This also fixes a separate issue with the pkg-config handling that could
cause partial application of pkg-config flags if the first part of the
pkg-config output parses correctly but there is an error later on. This
error isn't always fatal as we fall back to a plain -lfoo in the case of
linkSystemLibrary().
Previously it would fail as `renameW` do not ever fail with
`PathAlreadyExists`.
As a workaround we check for dest dir existence before rename
on Windows.
Compile error test cases can now be given as a sequence of files:
- "foo.1.zig"
- "foo.2.zig"
- "foo.3.zig"
- etc.
This sequence of files is tested as incremental compilation updates to a
single "foo.zig" source file.
To help avoid mistakes, we enforce strict ordering for these files.
"foo.zig" cannot co-exist with "foo.X.zig", the sequence must include
"foo.1.zig", and no numbers may be skipped.
In 008b0ec5e5 the `std.Thread.Mutex` API was changed
from `acquire` and `release` to `lock` and `unlock`. `std.event.Lock` still uses `acquire`
and `release`. `std.event.WaitGroup` is using `std.Thread.Mutex` and was not updated to use
`lock` and `unlock`, and so compilation failed prior to this commit.
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
* std: add Thread.Condition.timedWait
I needed the equivalent of `std::condition_variable::wait_for`, but it's missing in std.
This PR adds an implementation, following the status quo of using std.os.CLOCK.REALTIME in the pthread case (i.e. Futex)
A follow-up patch moving futex/condition stuff to monotonic clocks where available seems like a good idea.
This would involve conditionally exposing more functions and constants through std.c and std.os.
For instance, Chromium picks `pthread_cond_timedwait_relative_np` on macOS and `clock_gettime(CLOCK_MONOTONIC...)` on BSD's.
Tested on Windows 11, macOS 12.2.1 and Linux (with/without libc)
* Sleep in the single threaded case, handle timeout overflow in the Windows case and address a race condition in the AtomicCondition case.
This causes false positive "foo depends on itself" errors. Prior to some
recent enhancements, this type resolution was needed, however, we now
have a more sophisticated type resolution mechanism that fully
resolves types for the backend, but only after the Decl is fully
analyzed, avoiding dependency loops.
This is the x25519 counterpart to `edwards25519.clearCofactor()`.
It is useful to check for low-order points in protocols where it matters and where clamping cannot work, such as PAKEs.
This also addresses a nit from #10133 where IntT might be a confusing
name because it might imply signed integer (iX, not uX). We settled on
TBits for math/float.zig so I've applied that change here too.
When I originally wrote ldexp() I copied the name from parse_hex_float.
Previously, Zig would try to generate a function whose type contained
structs or unions which had not been fully resolved due to circular
dependency errors. With this commit, `resolveTypeFully` will be sure to
return `error.AnalysisFail` even in this scenario, leading to proper
display of compilation errors instead of a crash.
Rather than using blocks and control flow to check which operand is the maximum or minimum,
we use wasm's `select` instruction which returns us the operand based on a result from a comparison.
This saves us the need of control flow, as well as reduce the instruction count from 13 to 7.
Fixes#11353
The renderer treats comments and doc comments differently since doc
comments are parsed into the Ast. This commit adds a check after getting
the text for the doc comment and trims whitespace at the end before
rendering.
The `a = 0,` in the test is here to avoid a ParseError while parsing the
test.
I consider this an interim workaround/hack until #1299 is finished.
There is a bug in the original C implementation of the errol3 (and errol4)
algorithm that can result in undefined behavior or an obviously incorrect
result (leading ':' in the output)
This change checks for those two problems and uses a slower fallback
path if they occur. I can't guarantee that this will always produce
the correct result, but since the workaround is only used if the original
algorithm is guaranteed to fail, it should never turn a previously-correct
result into an incorrect one.
Fixes#11283
Add support for emitting debug info for local variables within a subprogram.
This required moving bits responsible for populating the debug info back to
`CodeGen` from `Emit` as we require the operand to be resolved at callsite
plus we need to know its type. Without enforcing this, we could end up
with a `dead` mcv.
Adds a function that allows checking for memory leaks (and other problems) by taking advantage of the FailingAllocator and inducing failure at every allocation point within the provided `test_fn` (based on the strategy employed in the Zig parser tests, which can now use this function).
* If more than one error is reported for the same Decl, the first error
message is kept and the second one discarded.
* Prevent functions from being sent to codegen backends if there were
any errors resolving any of their parameter types or return type.
This commit removes the tiny amount of dependency on async/await that
the self-hosted compiler has so that it can self-host before async/await
language features are working.
This shuffles some tests do ensure the new instructions are tested for the wasm backend,
by moving vectors into their own tests as well as move the f16 test cases as those require
special operating also.
Implements the `ctz` AIR instruction for integers with bitsize <= 64.
When the bitsize of the integer does not match the bitsize of a wasm type,
we first XOR the value with the value of (1<<bitsize) to set the right bits
and ensure we will only count the trailing zeroes of the integer with the correct bitsize.
Implements the `clz` AIR instruction for integers with bitsize <= 64.
When the bitsize of the integer is not the same as wasm's bitsize,
we substract the difference in bits as those will always be 0 for the integer, but should
not be counted towards the end result. We also wrap the result to ensure it fits
in the result type as documented in the language reference.
This implements the `mul_add` AIR instruction for floats of bitsize 32 and 64.
f16's will require us being able to extend and truncate f16's to correctly
store and load them without losing the accuracy.
This implements the `max` and `min` AIR instructions by checking
whether LHS is great/lesser than RHS. If that's the case, we assign
LHS to the result, otherwise assign RHS to it instead.
This way, if the user wants to use `codesign` (or other tool) they
will not be forced to `-f` force signature update. This matches
the behavior promoted by Apple's `ld64` linker.
See #11367
It's debatable whether this ends up being a legitimate compile error or
whether the lang spec allows this test case. For now this workaround
seems very reasonable; delaying comptime execution of `verifyContext`
until the struct is instantiated.
These are more efficiently semantically analyzed. More importantly, if
they don't match, we get a crash in Sema.
Missing places prior to this commit:
* labeled blocks
* `break` and `continue` on comptime (not inline) loops
* `if`, `try`, `orelse`, and `catch` inside comptime scopes
* `-Dskip-compile-errors` is removed; `-Dskip-stage1` is added.
* Use `std.testing.allocator` instead of a new instance of GPA.
- Fix the memory leaks this revealed.
* Show the file name when it is not parsed correctly such as when the
manifest is missing.
- Better error messages when test files are not parsed correctly.
* Ignore unknown files such as swap files.
* Move logic from declarative file to the test harness implementation.
* Move stage1 tests to stage2 tests where appropriate.
There was a simple missing check of adding an inferred error set to
itself, in which case we should not try to mutate the hash map while
iterating over it.
Sema avoids adding map entries for certain instructions such as
`set_eval_branch_quota` and `atomic_store`. This means that result
location semantics in AstGen must not emit any instructions that attempt
to use the result of any of these instructions.
This commit makes AstGen replace such instructions with
`Zir.Inst.Ref.void_value` if their result value ends up being
referenced.
This fixes a compiler crash when running std lib atomic tests.
Avoids many pitfalls connected with premature/early return in case
there are errors with Decl, etc. This is effectively bringing back
the old design however in a much nicer packaging, where every
mechanism related to tracking Decl's debug info is now nicely
wrapped in a single struct (aka the `DeclState`). This includes
relocation table, type arena, etc. It is now the caller's
responsibility to deinit the state (so that no memory is leaked)
after `Decl` has been analysed (or errored out). The caller here
is typically a linker such as `Elf` or `MachO`.
This is not complete support for asm expressions, but allows a few more
test cases from test/behavior/asm.zig to pass. Since the non-register
inputs are named `input_${n}` they can cause name collisions: I'm
wrapping the asm expressions in their own block to prevent that.
Contextually, this change also makes test/behavior/asm.zig run for
stage2, but skips individual tests for most backends (I only verified
the C and LLVM backends successfully run one new test case) and the
entire test file for aarch64, where it's running into preexisting
shortcomings.
Instead, use ResultLoc.none to allow for the expression type to be
inferred [^1]. This effectively moves the type coercion to Sema, in
order to turn comptime values into usable values for the backends to
consume. Right now the coercion is applies as comptime_int -> usize and
comptime_float -> f64, as an arbitrary choice.
[^1]: 9f25c8140c/src/AstGen.zig (L207-L208)
An assembly expression in a comptime block is legal Zig in the case of
global assembly [^1]. Instead of unconditionally asserting that the
expression lives in a runtime block, here we assert that if the
expression lives in a comptime block it must be outside of function
scope.
[^1]: https://ziglang.org/documentation/0.9.1/#Global-Assembly
* Sema: store the precomputed monomorphed_funcs hash inside Module.Fn.
This is important because it may be accessed when resizing monomorphed_funcs
while this Fn has already been added to the set, but does not have the
owner_decl, comptime_args, or other fields populated yet.
* Sema: in `analyzeIsNonErr`, take advantage of the AIR tag being
`wrap_errunion_payload` to infer that `is_non_err` is comptime true
without performing any error set resolution.
- Also add some code to check for empty inferred error sets in this
function. If necessary we do resolve the inferred error set.
* Sema: queue full type resolution of payload type when
`wrap_errunion_payload` AIR instruction is emitted. This ensures the
backend may check the alignment of it.
* Sema: resolveTypeFully now additionally resolves comptime-only
status.
closes#11306
This commit introduces a new AIR instruction `cmp_lt_errors_len`. It's
specific to this use case for two reasons:
* The total number of errors is not stable during semantic analysis; it
can only be reliably checked when flush() is called. So the backend
that is lowering the instruction must emit a relocation of some kind
and then populate it during flush().
* The fewer AIR instructions in memory, the better for compiler
performance, so we squish complex meanings into AIR tags without
hesitation.
The instruction is implemented only in the LLVM backend so far. It does
this by creating a simple function which is gutted and re-populated
with each flush().
AstGen now uses ResultLoc.coerced_ty for `@intToError` and Sema does the
coercion.
* In semaStructFields and semaUnionFields we return error.GenericPoison
if one of the field types ends up being generic poison.
- This requires handling function calls and function types taking
this into account when calling `typeRequiresComptime` on the return
type.
* Unrelated: I noticed using Valgrind that struct reification did not
populate the `known_opv` field. After fixing it, the behavior tests
run Valgrind-clean.
* ZIR: use `@ptrCast` to cast between slices instead of exploiting
the fact that stage1 incorrectly allows `@bitCast` between slices.
- A future enhancement will make Zig support `@ptrCast` to directly
cast between slices.
The init()/commit() API of this field leads to the type of bug that this
commit fixes by defering an uncomfortably complex expression. I didn't
bother doing the equivalent fix in link/MachO.zig because instead I
think the `decl_state` field should be entirely removed from Dwarf.
To no longer set the error code to undefined. This fixes the problem
where an undefined single-item pointer coerced to an error union of a
slice set the whole thing to undefined even though the sub-coercion to
the slice would have produced a defined value.
Since .rdi is not part of the callee saved registers, it needs to be
proactively spilled to the stack so that we don't clobber the
return address where to save the return value.
This is now required to correctly track and spill registers
required for some ops such `mul` or `div` (both required use of
`.rax` and `.rdx` registers).
Commit 052079c994 surfaced two issues with
the generated C code:
- renderInt128() contained a seemingly unnecessary assertion to verify
that the high 64 bits of the number were nonzero, dating back to
9bf1681990. I removed it.
- renderValue() didn't have any special handling for undefined structs,
falling back to printing "{}" which generated invalid expressions
such as "return {}" for functions returning structs, whereas
"return (S){}" is the correct form. I changed it accordingly.
At the same time I'm reenabling the relevant tests.
Similar code was already in place for conditional branches. This updates
AstGen to do the same for labeled blocks. It takes advantage of the
`store_to_block_ptr` instructions by mutating them in place to become
`as` instructions, coercing the break operands before they are returned
from the block.
Sema:
* queue full resolution of std.builtin.Type.Error when doing `@typeInfo`
for error sets.
LLVM backend:
* change a TODO comment to a proper explanation of why debug info
for structs is left as a fwd decl sometimes.
* remove handling of packed unions which does not match the type
information or constant generation code.
* remove copy+pasted code
* fix union debug info not matching the memory layout
* remove unnecessary error checks and type casting
* Added peer type resolution for arrays and vectors: the vector type is
selected.
* Fixed passing the lhs type or rhs type instead of the peer resolved
type when calling Value methods during analyzeArithmetic handling of
comptime expressions.
* `checkVectorizableBinaryOperands` now allows mixing vectors and
arrays, as long as one of the operands is a vector.
This matches stage1's handling of `^=` but apparently stage1 is
inconsistent and does not handle e.g. `*=`. stage2 now will always allow
mixing vector and array operands for all operations.
Although lazy_align is a different tag than integer values, it needs to
be hashed and equality-tested as if it were a simple integer. Otherwise
our basic data structure guarantees fall apart.
We are putting off actual optimization of floats because we have a
couple proposals being considered which would change how it works.
In the meantime, lowering optimized float mode to be the same as
strict is a perfectly legal way to implement the Zig language specification.
Otherwise, we risk tripping an assertion in `Type.errorSetNames()`
in case the inferred error set was not yet fully resolved. This so
far only surfaced when Dwarf triggers recursive analysis of an
error set as part of emitting debug info for an error union.
This implements the overflow arithmetic for unsigned and signed integers.
Meaning the following instructions:
- @addWithOverflow
- @subWithOverflow
- @shlWithOverflow
- @mulWithOverflow
RunStep on unexpected exit code used to return error.UncleanExit, which
was confusing and unclear. When it was changed, the error handling code
in build_runner was not modified, which produced an error trace.
This commit explicitly handles error.UnexpectedExitCode in build_runner
so that the behavior now matches that of zig 0.8.1 after which it was
regressed.
Instead, we fallback to the old-fashioned stabs-based mechanism
until I add the missing mechanism for extracting and relocating
DWARF from relocatable object files and writing it into a dSYM
bundle.
continuation of #11093 to simplify testing IPC
* use cases
- get path to temporary directory
- get the test arguments inside test block for reusage
- build executables from text within test blocks, ie to test IPC
* missing conventions
- how to name and debug test cases
- where do simple+repititve build commands for testing belong
This file contains a collections of functions that may be useful for SIMD, such as generating a vector with a linear range of numbers starting at zero, joining two vectors together, getting the index of the first true in a vector of bools, etc.
Rather than creating an import for externs on updateDecl, we now
generate them when they're referenced. This is required so using @TypeOf(extern_fn())
will not emit the import into the binary (causing an incorrect function type index
as it won't be fully analyzed).
add_with_overflow and similar functions now have the ty_pl data
attached. The Payload will now be a binary operation and the inst is
expected to return a tuple consisting of the destination integer type
and an overflow bit (u1).
Co-authored-by: Jan Philipp Hafer <jan.hafer@rwth-aachen.de>
* AstGen: restore the param_type ZIR instruction and pass it to the
expression for function call arguments. This does not solve the
problem for generic function parameters, but it catches stage2 up to
stage1 which also does not solve the problem for generic function
parameters.
- Most of the enhancements in this commit will still be needed for a
more sophisticated further improvement to handle generic function
types.
- In Sema, handling of `as` coercion recognizes the `var_args_param`
Type Tag and passes the operand through doing no coercion.
- That was the last ZIR tag and we are now using all 256 ZIR tags.
* AstGen: array init and struct init expressions use the anon form even
when the result location has a type. Prevents the type system
incorrectly believing, for example, that a tuple is actually an array
when the result location is a param_type of a function with `anytype`
parameter.
* Sema: add missing coercion in `unionInit` to coerce the init to the
corresponding union field type.
* `Value.fieldValue` now takes a type and does not take an allocator.
closes#11293
After this commit, stage2 passes all the parser tests.
Some cases had to stay behind, either because they required complex case
configuration that we don't support in independent files yet, or because
they have associated comments which we don't want to lose track of.
To make sure I didn't drop any tests in the process, I logged all
obj/test/exe test cases from a run of "zig build test" and compared
before/after this change.
All of the test cases match, with two exceptions:
- "use of comptime-known undefined function value" was deleted, since
it was a duplicate
- "slice sentinel mismatch" was renamed to "vector index out of
bounds", since it was incorrectly named
This change adds a "--exclude" parameter to zig format, which can be
used to make sure that it does not process certain files or folders
when recursively walking a directory.
To do this, we simply piggy-back on the existing "seen" logic in zig
fmt and mark these files/folders as seen before processing begins.
This brings two quality-of-life improvements for folks working on
compile error test cases:
- test cases can be added/changed without re-building Zig
- wrapping the source in a multi-line string literal is not necessary
I decided to keep things as simple as possible for this initial
implementation. The test "manifest" is a contiguous comment block at the
end of the test file:
1. The first line is the test case name
2. The second line is a blank comment
2. The following lines are expected errors
Here's an example:
```zig
const U = union(enum(u2)) {
A: u8,
B: u8,
C: u8,
D: u8,
E: u8,
};
export fn entry() void {
_ = U{ .E = 1 };
}
// union with too small explicit unsigned tag type
//
// tmp.zig:1:22: error: specified integer tag type cannot represent every field
// tmp.zig:1:22: note: type u2 cannot fit values in range 0...4
```
The mode of the test (obj/exe/test), as well as the target
(stage1/stage2) is determined based on the directory containing the
test.
We'll probably eventually want to support embedding this information
in the test files themselves, similar to the arocc test runner, but
that enhancement can be tackled later.
Closures are not necessarily constant values. For example, Zig
code might do something like this:
fn foo(x: anytype) void {
const S = struct {field: @TypeOf(x)};
}
...in which case the closure_capture instruction has access to a
runtime value only. In such case we preserve the type and use a
dummy runtime value.
closes#11292
This also fixes a bug that I didn't see causing any problems yet in
generic function instantiation where it would read from a GetOrPutResult
too late.
Also it delays full resolution of generic function type parameters until
after the function body is finished being analyzed.
closes#11291
The unit test for hasUniqueRepresentation asserted that a vector of
length 3 would not have a unique representation. This would be true if
it were lowered to ABI size 8 instead of 6. However lowering it to ABI
size 6 is perfectly valid depending on the target.
This commit also simplifies the logic for hasUniqueRepresentation of
integers.
`const foo = comptime ...` generated invalid ZIR when the initialization
expression contained an array literal because the
validate_array_init_comptime instruction assumed that the corresponding
alloc instruction was comptime. The solution is to look slightly ahead
and notice that the initialization expression would be comptime-known
and affect the alloc instruction tag accordingly.
The code for detecting when a local const initialization expression
ended up being comptime-known gave up when it encountered dbg_stmt
instructions, but such instructions are not supposed to matter.
The unit tests of std.meta depended on `@typeInfo` for the same type
returning a slice of declarations and fields with the same pointer
address. This is not something guaranteed by the language specification.
* std.meta: correct use of `default_value` in reification. stage1
accepted a wrong type for `null`.
* Sema: after instantiating a generic function, if the return type ends
up being a comptime-known type, then we return an error, undoing the
generic function instantiation, and making a comptime function call
instead.
- We also needed to clean up the dependency graph in this case.
* Sema: reified enums set tag_ty_inferred to false since an integer tag
type is provided. This is a limitation of the `@Type` builtin which
will be addressed with #10710.
* Sema: fix resolveInferredErrorSet incorrectly calling
ensureFuncBodyAnalyzed on generic functions.
For Value.Tag.bytes, the value copy implementation did not copy the
bytes array. No good. This operation must do a deep copy. If we want
some other mechanism for not copying very large byte buffers then it has
to work differently than this one.
That happens after a function body is analyzed. This prevents circular
dependency compile errors and yet a way to mark types that need to be
fully resolved before a given function is sent to the codegen backend.
The runtime behavior allowed this in both stage1 and stage2, but stage1
fails with index out of bounds during comptime. This behavior makes
sense to support, and comptime behavior should match runtime behavior. I
implement this fix only in stage2.
My previous commit added a new behavior test that passes for stage2 but
I forgot to check whether it passes for stage1. Since it does not, it
has to be disabled.
Additionally, this commit organizes behavior tests; there is no longer a
section of tests only passing for stage1. Instead, tests are disabled on
an individual basis. There is an except for the file which has global
assembly in it.
We were enforcing bounds on the index of an elem_ptr in pointerDeref,
but we want to support out-of-bounds accesses by reinterpreting memory.
This removes that check, so that the deref falls back to bitcasting, as
usual.
This was masked by another bug that was forcing bitcasts incorrectly,
which is why this wasn't noticed earlier.
Sema.pointerDeref() assumes that elem_ptrs have been "flattened" when
they were created, so that you an elem_ptr will never be the array_ptr
of another elem_ptr when they share the same type.
Value.elemPtr does this already, but a couple of places in Sema were
bypassing this logic.
All tests have been manually verified which are now passing. This means that any remaining
TODO is an actual to-be-fixed or to-be-implemented test case.
Implements lowering constants for pointers of value 'opt_payload_ptr'.
The offset is calculated by determining the abi size of the full type and
then substracting the payload size.
Error sets contain the entire global error set.
Users are often switching on specific errors only present within that operand.
This means that cases are almost always sparse and not contiguous.
For this reason, we will instead emit the default case for error values not present in
that specific operand error set. This is fine as those cases will never be hit,
as prevented by the type system.
By still allowing jump tables for those cases, rather than if-else chains, we save runtime cost
as well as binary size.
This implements the `error_name` instruction, which is emit for runtime `@errorName` callsites.
The implementation works by creating 2 symbols and corresponding atoms.
The initial symbol contains a table which each element consisting of a slice where the ptr field
points towards the error name, and the len field contains the error name length without the sentinel.
The secondary symbol contains a list of all error names from the global error set.
During the error_name instruction, we first get a pointer to the first symbol.
Then based on the operand we perform pointer arithmetic, to get the correct index into this table.
e.g. error index 2 = ptr + (2 * ptr size). The result of this will be stored in a local
and then returned as instruction result.
During `flush()` we populate the error names table by looping over the global error set
and creating a relocation for each error name. This relocation is appended to the table symbol.
Then finally, this name is written to the names list itself.
Finally, both symbols' atom are allocated within the rest of the binary.
When no error name is referenced, the `error_name_symbol` is never set, and therefore
no error name table will be emit into the final binary.
This includes various fixes/improvements to the C backend to improve
error/union support. It also fixes up our handling of decls, where some
decls were not correctly marked alive.
This flag is used when building stage1 to omit the stage2 backends from
the compiler to save memory on the CI server. It regressed with the
merging of e8813b296b because Value
functions started calling into Sema functions.
The end goal for this build option is to eliminate it.
This is from discussions from #11249. The stage2 behavior is correct and
is strictly more accurate, so we'd prefer to keep it. In that case, I
modified the behavior tests to have the conditional between
stage1/stage2 and get this test passing.
This commit adds a new optional argument to several Value methods which
provides the ability to resolve types if it comes to it. This prevents
having duplicated logic inside both Sema and Value.
With this commit, the "struct contains slice of itself" test is passing
by exploiting the new lazy_align Value Tag.
This reverts commit 3701697a0a.
The commit introduced a regression when building stage2 on nixOS where
the linker would fail to find relevant LLVM dynamic libraries as some
search dirs were missing.
With this change, we can now bake in entitlements into the binary.
Additionally, I see this as the first step towards full code signature
support which includes baking in Apple issued certificates for
redistribution, etc.
Also update std/build.zig to use stage2 function pointer semantics.
This gets us a little bit closer to `zig build` working, although it is
now hitting a new crash in the compiler.
* Use `@Vector` syntax instead of `std.meta.Vector`.
* Use `var` instead of `const` for tests so that we get runtime
coverage instead of only comptime coverage. Comptime coverage is done
with `comptime doTheTest()` calls.
Made most `Value` functions require a `Type`. If the provided type is a
vector, then automatically vectorize the operation and return with
another vector. The Sema side can then automatically become vectorized
with minimal changes. There are already a few manually vectorized
instructions, but we can simplify those later.
The existing `cmp_*` instructions get their result type from `lhs`, but
vector comparison will always return a vector of bools with only the
length derived from its operands. This necessitates the creation of a
new AIR instruction.
the 3 tests that called `testArray2DConstDoublePtr` started passing
after implementing `ptr_elem_val`. the rest of these I think were
already passing before.
The codegen for this is almost identical to `ptr_elem_ptr` except
there's an extra `mov` at the end to replace the pointer with the
value it points to, "in-place" (which can be done in a single
instruction without any extra registers).
This implements improvements/fixes to get all the union, tuple, and array behavior tests passing.
Previously, we lowered parent pointers for field_ptr and element_ptr incompletely. This has now
been improved to recursively lower such pointer.
Also a fix was done to `generateSymbol` when checking a container's layout.
Previously it was assumed to always be a struct. However, the type can also be a tuple, and therefore
panicking. Updating to ask a type's container layout instead allows us to keep a singular branch for both cases.
Example scenario:
test {
const a: i32 = blk: {
if (false) break :blk 24;
};
_ = a;
}
Prior to this, this would panic the compiler with a source needed error.
This provides the source as `sema.src`. This is not ideal, since the
line it points to is pretty far from the true issue. (One block out)
But, this prevents the compiler from straight up crashing and follows a
pattern used by similar ZIR which don't provide a src loc.
Add placeholder files for Codegen, Emit, and Mir stages, complete with
a placeholder implementation of generate() to make it able to be plugged in
to the frontend. At the moment the implementation just panics, it'll be
worked on incrementally later.
Also, this registers the sparcv9 backend files into CMakeLists.txt.
* Identify the ones that are passing and stop skipping them.
* Flatten out the main behavior.zig file and have each individual test
disable itself if it is not passing.
This function took a parameter that was only ever used with one value,
obscuring the fact that it was a regular `block` which should be used with
`.break` and not `.break_inline`.
* make it always return a fully qualified name. stage1 is inconsistent
about this.
* AstGen: fix anon_name_strategy to correctly be `func` when anon type
creation happens in the operand of the return expression.
* Sema: implement type names for the "function" naming strategy.
* Put "enum", "union", "opaque", or "struct" in place of "anon" when
creating respective anonymous Decl names.
* std.testing: add `expectStringStartsWith`. Didn't end up using it
after all.
Also this enables the real test runner for stage2 LLVM backend (sans
wasm32) since it works now.
Instead of doing it before the switch tower, do it afterwards, so that
special handling may be done before undefined gets casted to the
destination type.
In this case the special handling we want to do is *[N]T to []T setting
the slice length based on the array length, even when the array value is
undefined.
When a generic call evaluates to a generic type, the call will be re-generated.
However, the old function was not freed before being re-generated, causing a memory leak.
So rather than only returning an error, we first free the old value.
Notably, Value.eql and Value.hash are improved to treat NaN as equal to
itself, so that Type/Value can be hash map keys. Likewise float hashing
normalizes the float value before computing the hash.
When the length is comptime-known, we perform an inline loop instead of emitting
a runtime loop into the binary.
This also allows us to easily write 'undefined' to aggregate types.
We now do this when we set the error tag of an error union where the payload will be set to undefined.
This implements the `memcpy` instruction and also updates the inline memcpy calls
to make use of the same implementation. We use the fast-loop when the length is comptime known,
and use a runtime loop when the length is runtime known.
We also perform feature-dection to emit a simply wasm memory.copy instruction when the feature
'bulk-memory' is enabled. (off by default).
This reverts commit 8e7b1a74ac.
Sorry, I should have put up a PR and ran that one by Jakub and done some
more inspection.
This causes problems with gdb:
BFD: /home/andy/dev/zig/build-release/test: invalid string offset 3254779904 >= 153524 for section `.shstrtab'
This fixes lack of stack traces on arm64 macOS which were regressed
and not getting generated at all after this addition to write
current stack traces. Prior to this, function `isValidMemory` would
sync two subsequent pages if the aligned (base) address was different
than the frame pointer. I fail to see what the logic for such assumption
here is as the manual of `msync` clearly states it will fail with error
if the passed in memory region length contains unmapped regions.
This was the very reason why there were no stack traces print on
arm64 macOS as the second page was unmapped thus incorrectly flagging
the frame pointer as invalid.
* Sema: fix `zirTypeInfo` allocating with the wrong arenas for some
stuff.
* LLVM: split `airDbgInline` into two functions, one for each AIR tag.
- remove the redundant copy to type_map_arena. This is the first
thing that lowerDebugType does so this hack was probably just
accidentally avoiding UB (which is still present prior to this
commit).
- don't store an inline fn inst into the di_map for the generic
decl.
- use a dummy function type for the debug info to avoid whatever UB
is happening.
- we are now ignoring the function type passed in with the
dbg_inline_begin and dbg_inline_end.
* behavior tests: prepare the vector tests to be enabled one at a time.
Mitigates #11199.
`testing.expect` is better than `testing.expectEqual` for behavior
tests. Better for behavior tests to stick to only testing the limited
behavior they are meant to test and avoid functions such as
`expectEqual` that drag in too much of the standard library (in this
case to print helpful diffs about why a value is not equal to another).
Adds the sentinel element to the type name to avoid ambiguous
declarations, and outputs the sentinel element (if needed) even in what
would otherwise be empty arrays.
I think that reusing the ComptimePtrLoad infrastructure is ultimately
less logic and more robust than adding a `direct` flag to elem_ptr.
* Some code in zirTypeInfo needed to be fixed to create proper
Type/Value encodings.
* comptime elemVal works by constructing an elem_ptr Value and then
using the already existing pointerDeref function.
There are some remaining calls to Value.elemValue which should be
considered code smells at this point.
This fixes one of the major issues plaguing the `std.sort` comptime tests.
The high level issue is that at comptime, we need to know whether `elem_ptr` is
being used to subslice an array-like pointer or access a child value. High-level
example:
var x: [2][2]i32 = undefined;
var a = &x[0]; // elem_ptr, type *[2]i32
var y: [5]i32 = undefined;
var b = y[1..3]; // elem_ptr, type *[2]i32
`a` is pointing directly to the 0th element of `x`. But `b` is
subslicing the 1st and 2nd element of `y`. At runtime with a well
defined memory layout, this is an inconsequential detail. At comptime,
the values aren't laid out exactly in-memory so we need to know the
difference.
This becomes an issue specifically in this case:
var c: []i32 = a;
var d: []i32 = b;
When converting the `*[N]T` to `[]T` we need to know what array to point
to. For runtime, its all the same. For comptime, we need to know if its
the parent array or the child value.
See the behavior tests for more details.
This commit fixes this by adding a boolean to track this on the
`elem_ptr`. We can't just immediately deref the child for `&x[0]`
because it is legal to ptrCast it to a many-pointer, do arithmetic, and
then cast it back (see behavior test) so we need to retain access to the
"parent" indexable.
Currently, the new API will only be available on macOS with
the intention of adding more POSIX systems to it incrementally
(such as Linux, etc.).
Changes:
* add `posix_spawn` wrappers in a separate container in
`os/posix_spawn.zig`
* rewrite `ChildProcess.spawnPosix` using `posix_spawn` targeting macOS
as `ChildProcess.spawnMacos`
* introduce a `posix_spawn` specific `std.c.waitpid` wrapper which
does return an error in case the child process failed to exec - this
is required for any process that was spawned using `posix_spawn`
mechanism as, by definition, the errors returned by `posix_spawn`
routine cover only the `fork`-equivalent; `pre-exec()` and `exec()`
steps are covered by a catch-all error `ECHILD` returned by `waitpid`
on unsuccessful execution, e.g., no such file error, etc.
This adds the GPR/FPR register definitions and instruction formats
for SPARCv9.
I need to implement a separate enc() function because the register
values for the FPRs have to be encoded to a special format that's
separate from the normal register ID.
This involved some significant reworking in order to introduce the
concept of "forward declarations" to the system to break dependency
loops.
The `lowerDebugType` function now takes an `enum { full, fwd }` and is moved
from `DeclGen` to `Object` so that it can be called from `flushModule`.
`DITypeMap` is now an `ArrayHashMap` instead of a `HashMap` so that we can
iterate over the entries in `flushModule` and finalize the forward decl
DITypes into full DITypes.
`DITypeMap` now stores `AnnotatedDITypePtr` values instead of
`*DIType` values. This is an abstraction around a `usize` which assumes
the pointers will be at least 2 bytes aligned and uses the least
significant bit to store whether it is forward decl or a fully resolved
debug info type.
`lowerDebugTypeImpl` is extracted out from `lowerDebugType` and it has a
mechanism for completing a forward decl DIType to a fully resolved one.
The function now contains lowering for struct types. Closes#11095.
There is a workaround for struct types which have not had
`resolveFieldTypes` called in Sema, even by the time `flushModule` is
called. This is a deficiency of Sema that should be addressed, and the
workaround removed. I think Sema needs a new mechanism to queue up type
resolution work instead of doing it in-line, so that it does not cause
false dependency loops. We already have one failing behavior test
because of a false dependency loop.
Prior to these, the return type was non-null but the value was generic
poison which wasn't usable in user-space. This sets the value to null.
This also adds a behavior test for this.
Co-authored-by: InKryption <inkryption07@gmail.com>
This resolves https://github.com/ziglang/zig/issues/11159
The problem was that:
1. We were not correctly deleting the field stores after recognizing
that an array initializer was a comptime-known value.
2. LLVM was not checking that the final type had no runtime bits, and
so would generate an invalid store.
This also adds several test cases for related bugs, just to check these
in for later work.
We must resolve the type fully so that pointer children (i.e. slices)
are resolved. Additionally, we must resolve even if we can know the
value at comptime because the `alloc_inferred` ZIR always produces a
constant in the AIR.
Fixes#11181
A const local which had its init expression write to the result pointer,
but then gets elided to directly initialize, was missing the coercion to
the type annotation.
DWARF 5 moves around some fields and adds a few new ones that can't be
parsed or ignored by our current DWARF 4 parser. This isn't a complete
implementation of DWARF 5, but this is enough to make stack traces
mostly work. Line numbers from C++ don't show up, but I know the info
is there. I think the answer is to iterate through .debug_line_str in
getLineNumberInfo, but I didn't want to fall into an even deeper rabbit
hole tonight.
Compilers will sometimes `#define linux 1` if the operating system in
use is Linux. This clashes with the code produced by the C backend when
processing the stdlib, e.g. std.Target.Os.VersionRange [^1] which is a
struct containing a field named `linux`.
The output of the C backend doesn't rely on this macro being defined,
and other code also shouldn't rely on it -- e.g. quoting from the GCC
documentation [^2]:
"""
The C standard requires that all system-specific macros be part of
the reserved namespace. All names which begin with two underscores, or
an underscore and a capital letter, are reserved for the compiler and
library to use as they wish. However, historically system-specific
macros have had names with no special prefix; for instance, it is
common to find unix defined on Unix systems.
[...]
We are slowly phasing out all predefined macros which are outside the
reserved namespace. You should never use them in new programs, and we
encourage you to correct older code to use the parallel macros
whenever you find it. We don’t recommend you use the system-specific
macros that are in the reserved namespace, either. It is better in the
long run to check specifically for features you need
"""
[^1]: 8c32d989c9/lib/std/target.zig (L224)
[^2]: https://gcc.gnu.org/onlinedocs/cpp/System-specific-Predefined-Macros.html#System-specific-Predefined-Macros
target.Arch already supports finding the correct encoding for either
target, so being able to do the inverse has use cases for when parsing
files of an unknown target (i.e. for zar).
* don't store `has_well_defined_layout` in memory.
* remove struct `hasWellDefinedLayout` logic. it's just
`layout != .Auto`. This means we only need one implementation, in
Type.
* fix some of the cases being wrong in `hasWellDefinedLayout`, such as
optional pointers.
* move `tag_ty_inferred` field into a position that makes it more
obvious how the struct layout will be done. Also we don't have a
compiler that intelligently moves fields around so this layout is
better.
* Sema: don't `resolveTypeLayout` in `zirCoerceResultPtr` unless
necessary.
* Rename `ComptimePtrLoadKit` `target` field to `pointee` to avoid
confusion with `target`.
We need to make sure that we bitcast our pointers correctly before
we use get_element_ptr to compute the offset for the parent
pointer.
This also includes a small fix-up for a problem where ptrs to const
i64/u64 were not using the correct type in >1-level decl chains
(where we call lowerParentPtr recursively)
The core change here is that we no longer blindly trust that parent
pointers (.elem_ptr, .field_ptr, .eu_payload_ptr, .union_payload_ptr)
were derived from the "true" type of the underlying decl. When types
diverge, direct dereference fails and we are forced to bitcast, as
usual.
In order to maximize our chances to have a successful bitcast, this
includes several changes to the dereference procedure:
- `root` is now `parent` and is the largest Value containing the
dereference target, with the condition that its layout and the
byte offset of the target within are both well-defined.
- If the target cannot be dereferenced directly, because the
pointers were not derived from the true type of the underlying
decl, then it is returned as null.
- `beginComptimePtrDeref` now accepts an optional array_ty param,
which is used to directly dereference an array from an elem_ptr,
if necessary. This allows us to dereference array types without
well-defined layouts (e.g. `[N]?u8`) at an offset
The load_ty also allows us to correctly "over-read" an .elem_ptr to an
array of [N]T, if necessary. This makes direct dereference work for
array types even in the presence of an offset, which is necessary if
the array has no well-defined layout (e.g. loading from `[6]?u8`)
This follows the same strategy as sema.typeRequiresComptime() and
type.comptimeOnly(): Two versions of the function, one which performs
resolution just-in-time and another which asserts that resolution is
complete.
Thankfully, this doesn't cause very viral type resolution, since
auto-layout structs and unions are very common and are known to not have
a well-defined layout without resolving their fields.
LLVM backend: generate DIGlobalVariable's for non-function globals and
rename linkage names when exporting functions and globals.
zig_llvm.cpp: add some wrappers to convert a handful of DI classes
into DINode's since DIGlobalVariable is not a DIScope like the others.
zig_llvm.cpp: add some wrappers to allow replacing the LinkageName of
DISubprogram and DIGlobalVariable.
zig_llvm.cpp: fix DI class mixup causing nonsense reinterpret_cast.
The end result is that GDB is now usable since you now no longer need
to manually cast every global nor fully qualify every export.
It is possible for the value length to be longer than the type because
we allow in-memory coercing of types such as `[5:0]u8` to `[5]u8`. In
such a case, the value length is 6 but the type length if 5.
The `.repeated` value type already got this right, so this is extending
similar logic out to `.aggregate` and `.bytes`. Both scenarios are
tested in behavior tests.
Fixes#11165
This also surfaces the fact that clz, ctz and popCount didn't actually
support 128 bit integers, despite what was claimed by
226fcd7c70. This was partially hidden by
the fact that the test code for popCount only exercised 128 bit integers
in a comptime context. This commit duplicates that test case for runtime
ints too.
Implements `@Type` for structs, anon structs, and tuples. This is another place that would probably benefit from a `.reified_struct` type tag but will defer for later in the interest of getting tests passing first.
Detect if we are storing an array operand to a bitcasted vector pointer.
If so, we instead reach through the bitcasted pointer to the vector pointer,
bitcast the array operand to a vector, and then lower this as a store of
a vector value to a vector pointer. This generally results in better code,
as well as working around an LLVM bug.
See #11154
This parameter is only currently needed by zig_byte_swap() and
zig_bit_reverse(). This commit adds an option to airBuiltinCall() to
allow emitting the signedness information only when needed, removing
this unused parameter from the other builtins.
This should cover not only integers, as done in
87744a7ea9, but also void, enums with a
single field, etc...
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
The libc interface uses `stat` instead of `stat64` struct.
This fixes, among other things, `zig fmt` accidentally setting the
formatted file's permission to 000.
This folds the airCountZeroes() code from
226fcd7c70 back into airBuiltinCall(),
since most of these builtins happen to require the same arguments and
can be unified under a common function signature.
This was already done for void types, and needs to be done for 0 bit
integer types as well to align the rendered function signatures with the
effective size of extra.data.args_len as seen by airCall().
Adds 2 new AIR instructions:
* dbg_var_ptr
* dbg_var_val
Sema no longer emits dbg_stmt AIR instructions when strip=true.
LLVM backend: fixed lowerPtrToVoid when calling ptrAlignment on
the element type is problematic.
LLVM backend: fixed alloca instructions improperly getting debug
location annotated, causing chaotic debug info behavior.
zig_llvm.cpp: fixed incorrect bindings for a function that should use
unsigned integers for line and column.
A bunch of C test cases regressed because the new dbg_var AIR
instructions caused their operands to be alive, exposing latent bugs.
Mostly it's just a problem that the C backend lowers mutable
and const slices to the same C type, so we need to represent that in the
C backend instead of printing two duplicate typedefs.
In stage1, this behavior was allowed (by accident?) and also
accidentally exercised by the behavior test changed in this commit. In
discussion on Discord, Andrew decided this should not be allowed in
stage2 since there is currently on real world reason to allow this
strange edge case.
I've added the compiler test to solidify that this behavior should NOT
occur and updated the behavior test to the new valid semantics.
- Cli operations should be refactored, since the standard test runner
has an expected argument structure. This would also ensure that the
test cli is usable as tested library with checks for subprocess
error or success instead of "hacky shell script interfaces".
- Default paths generation based on tmpDir would also be useful.
- Anonymous pipes on windows are generated from named pipes
- Async IO does not work on anonymous pipes
- Remove finished TODO
Introduce `Module.ensureFuncBodyAnalyzed` and corresponding `Sema`
function. This mirrors `ensureDeclAnalyzed` except also waits until the
function body has been semantically analyzed, meaning that inferred
error sets will have been populated.
Resolving error sets can now emit a "unable to resolve inferred error
set" error instead of producing an incorrect error set type. Resolving
error sets now calls `ensureFuncBodyAnalyzed`. Closes#11046.
`coerceInMemoryAllowedErrorSets` now does a lot more work to avoid
resolving an inferred error set if possible. Same with
`wrapErrorUnionSet`.
Inferred error set types no longer check the `func` field to determine if
they are equal. That was incorrect because an inline or comptime function
call produces a unique error set which has the same `*Module.Fn` value for
this field. Instead we use the `*Module.Fn.InferredErrorSet` pointers to
test equality of inferred error sets.
* use the real start code for LLVM backend with x86_64-linux
- there is still a check for zig_backend after initializing the TLS
area to skip some stuff.
* introduce new AIR instructions and implement them for the LLVM
backend. They are the same as `call` except with a modifier.
- call_always_tail
- call_never_tail
- call_never_inline
* LLVM backend calls hasRuntimeBitsIgnoringComptime in more places to
avoid unnecessarily depending on comptimeOnly being resolved for some
types.
* LLVM backend: remove duplicate code for setting linkage and value
name. The canonical place for this is in `updateDeclExports`.
* LLVM backend: do some assembly template massaging to make `%%`
rendered as `%`. More hacks will be needed to make inline assembly
catch up with stage1.
* Reduce branching in Type.eql and Type.hash for error sets.
* `Type.eql` uses element-wise bytes comparison since it can rely on
the error sets being pre-sorted.
* Avoid unnecessarily skipping tests that are passing.
This implements type equality for error sets. This is done
through element-wise error set comparison.
Inferred error sets are always distinct types and other error sets are
always sorted. See #11022.
This also adds `std.sort.sortContext` and
`std.sort.insertionSortContext` which are more advanced methods that
allow overriding the `swap` method. The former calls the latter for now
because reworking the main sort implementation is a big task that can be
done later without any changes to the API.
When the anytype parameter had only one-possible-value
(e.g. `void`), it would create a mismatch in the function type and the
function call. Now the function type and the callsite both omit the
one-possible-value anytype parameter in instantiated generic functions.
This implements the initial fptrunc instruction. For all other floating-point truncating,
a call to compiler-rt is required. (This also updates fpext to emit the same error).
- Implement switching over booleans and pointers.
- Fix sparse-detection where the lowest value was never truly set
as it started at a non-zero number and the case was > 50.
- Fix indexing the jump table by ensuring it starts indexing from 0.
This required adjusting `Type.nameAlloc` to be used with a
general-purpose allocator and added `Type.nameAllocArena` for the arena
use case (avoids allocation sometimes).
Previously, we did this so that we could insert a debug variable
declaration intrinsic on the alloca. But there is a dbg.value intrinsic
for declaring variables that are values.
The comment notes that we can't because of constness, but the recent
work by @Vexu lets us use the bitcast ty cause constness comes later.
The following tests pass from this:
```
test "A" {
const ary = [_:0]u8{42};
const ptr: [*:0]const u8 = &ary;
try expect(ptr[1] == 0);
}
test "B" {
comptime {
const ary = [_:0]u8{42};
const ptr: [*:0]const u8 = &ary;
try expect(ptr[1] == 0);
}
}
```
By the time zirElemVal is reached for a many pointer, a load has already
happened, making sure the operand is already dereferenced.
This makes `mem.sliceTo` now work.
This uses a new ZIR inst `array_init_sent` (and a ref equivalent) to
represent array init expressions that terminate in a a sentinel value.
The sentienl value is the last value in the `MultiOp` payload. This
makes it a bit more awkward to deal with (lots of "len - 1") but makes
it so that the payload matches the fact that sentinels appear at the end
of arrays. However, this is not a hill I want to die on so if we want to
change it to index 0, I'm happy to do so.
This makes the following work properly:
try expect(@TypeOf([_:0]u8{}) == [0:0]u8);
Array types with sentinels were not being typed correctly in the
translation from ZIR to Sema (comptime). This modifies the `array_init`
ZIR to also retain the type of the init expression (note: untyped array
initialization is done via the `array_init_anon` ZIR and so is unchanged
in this commit).
* mul_add AIR instruction: use `pl_op` instead of `ty_pl`. The type is
always the same as the operand; no need to waste bytes redundantly
storing the type.
* AstGen: use coerced_ty for all the operands except for one which we
use to communicate the type.
* Sema: use the correct source location for requireRuntimeBlock in
handling of `@mulAdd`.
* native backends: handle liveness even for the functions that are
TODO.
* C backend: implement `@mulAdd`. It lowers to libc calls.
* LLVM backend: make `@mulAdd` handle all float types.
- improved fptrunc and fpext to handle f80 with compiler-rt calls.
* Value.mulAdd: handle all float types and use the `@mulAdd` builtin.
* behavior tests: revert the changes to testing `@mulAdd`. These
changes broke the test coverage, making it only tested at
compile-time.
Improved f80 support:
* std.math.fma handles f80
* move fma functions from freestanding libc to compiler-rt
- add __fmax and fmal
- make __fmax and fmaq only exported when they don't alias fmal.
- make their linkage weak just like the rest of compiler-rt symbols.
* removed `longDoubleIsF128` and replaced it with `longDoubleIs` which
takes a type as a parameter. The implementation is now more accurate
and handles more targets. Similarly, in stage2 the function
CTypes.sizeInBits is more accurate for long double for more targets.
This fixes 2 entrypoints within the self-hosted wasm linker that would be called
for the llvm backend, whereas we should simply call into the llvm backend to perform such action.
i.e. not allocate a decl index when we have an llvm object, and when flushing a module,
we should be calling it on llvm's object, rather than have the wasm linker perform the operation.
Also, this fixes the wasm intrinsics for wasm.memory.size and wasm.memory.grow.
Lastly, this commit ensures that when an extern function is being resolved, we tell LLVM how
to import such function.
Like decl code generation, also unify the wasm backend and the wasm linker to call into
the general purpose `codegen.zig` to generate the code for a function.
This also unifies the wasm backend to use `generateSymbol` when lowering a constant
that cannot be lowered to an immediate value.
As both decls and constants are now refactored, the old `genTypedValue` is removed.
To unify the wasm backend with the other backends, we will now call `generateSymbol` to
lower a Decl into bytes. This means we also have to change some function signatures
to comply with the linker interface.
Since the general purpose generateSymbol is less featureful than wasm's, some tests are
temporarily disabled.
When an union had a zero-sized payload type, we would lower the tag twice. This is fixed
by exiting early when `payload_size` is 0.
With regards to error unions, we were only accounting for padding for the payload field.
However, the errorset value can have a smaller alignment than the payload as well, i.e. error!usize.
We fix this by also accounting for padding/alignment of the error set tag of an error union.
This makes the following work properly (as it does in stage1, too):
var zero_ptr: [*:0]const u8 = undefined;
var no_zero_ptr: [*]const u8 = zero_ptr;
Prior to this this would fail with an "expected type" error since
coercion failed.
This brings back #10950, which was reverted in 5ab5e2e673
because it [introduced a regression in `zig run`](https://github.com/ziglang/zig/pull/10950#issuecomment-1049481212)
where the runtime arguments passed were incorrect.
I've fixed the issue, and notably this was the only location where we
directly relied on accessing arguments by index in this code still (all
other locations use the iterator proper) and so we should be all good to
go now.
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
stage1 peer resolves the given test to `[*c]u8` but stage2 makes that a
const u8. I believe stage2 behavior is correct since the pointer itself
is const.
Before this we would see ZIR code like this:
```
%69 = alloc_inferred_mut()
%70 = array_base_ptr(%69)
%71 = elem_ptr_imm(%70, 0)
```
This would crash the compiler because it expects to see a
`coerce_result_ptr` instruction after `alloc_inferred_mut`, but that
does not happen in this case because there is no type to coerce the
result pointer to.
In this commit I modified AstGen so that it has similar codegen as when
using a const instead of a var:
```
%69 = alloc_inferred_mut()
%76 = array_init_anon(.{%71, %73, %75})
%77 = store_to_inferred_ptr(%69, %76)
```
This does not obey result locations, meaning if you call a function
inside the initializer, it will end up doing a copy into the LHS.
Solving this problem, or changing the language to make this legal,
will be left for my future self to deal with. Hi future self!
I see you reading this commit log. Hope you're doing OK buddy.
Sema for `store_ptr` of a tuple where the pointer is in fact the same
element type as the operand had an issue where the comptime fields would
get incorrectly lowered to runtime stores to bogus addresses. This is
solved with an exception to the optimization in Sema for storing
pointers that handles tuples element-wise. In the case that we are
storing a tuple to itself, it skips the optimization. This results in
better code and avoids the problem. However this caused a regression in
GeneralPurposeAllocator from the standard library.
I regressed the test runner code back to the simpler path. It's too
hard to debug standard library code in the LLVM backend right now since
we don't have debug info hooked up. Also, we didn't have any behavior
test coverage of whatever was regressed, so let's try to get that
coverage added as a stepping stone to getting the standard library
working.
The checks detecting such no-op branches (essentially instructions
that branch to the instruction immediately following the branch) were
tightened to catch more of these occurrences.
Now it's centered around a switch on the chosen type tag which gives us
easy access to pointer data.
The logic is simplied and in some cases logic is removed when it is
sufficient to choose the type that is a better coercion target without
knowing whether such coercion will succeed ahead of time.
A bug is fixed at the bottom of the function; we were doing the opposite
of what we were supposed to with `seen_const`.
Also the bottom of the function has a more complete handling of the
possible combinations of `any_are_null`, `convert_to_slice`, and
`err_set_ty`.
In the behavior tests, not as many backends needed to be skipped.
* AIR: use pl_op instead of ty_pl for wasm_memory_size. No need to
store the type because the type is always `u32`.
* AstGen: use coerced_ty for `@wasmMemorySize` and `@wasmMemoryGrow`
and do the coercions in Sema.
* Sema: use more accurate source locations for errors.
* Provide more information in the compiler error message.
* Codegen: use liveness data to avoid lowering unused
`@wasmMemorySize`.
* LLVM backend: add implementation
- I wasn't able to test it because we are hitting a linker error for
`-target wasm32-wasi -fLLVM`.
* C backend: use `zig_unimplemented()` instead of silently doing wrong
behavior for these builtins.
* behavior tests: branch only on stage2_arch for inclusion of the
wasm.zig file. We would change it to `builtin.cpu.arch` but that is
causing a compiler crash on some backends.
This implements the wasm builtins by lowering to builtins that are supported by c-compilers.
In this case: Clang.
This also simplifies the `AIR` instruction as it now uses the payload field of `ty_pl` and `pl_op`
directly to store the index argument rather than storing it inside Extra. This saves us 4 bytes
per builtin call.
Similarly to the other wasm builtin, this implements the grow variation where the memory
index is a comptime known value. The operand as well as the result are runtime values.
This also verifies during semantic analysis the target we're building for is wasm, or else
emits a compilation error. This means that other backends do not have to handle this AIR instruction,
other than the wasm and LLVM backends.
This implements the `wasmMemorySize` builtin, in Sema and the Wasm backend.
The Stage2 implementation differs from stage1 in the way that `index` must be a comptime value.
The stage1 variant is incorrect, as the index is part of the instruction encoding, and therefore,
cannot be a runtime value.
This adds a special CWD file descriptor, AT.FDCWD (-2), to refer to the
current working directory. The `*at(...)` functions look for this and
resolve relative paths against the stored CWD. Absolute paths are
dynamically matched against the stored Preopens.
"os.initPreopensWasi()" must be called before std.os functions will
resolve relative or absolute paths correctly. This is asserted at
runtime.
Support has been added for: `open`, `rename`, `mkdir`, `rmdir`, `chdir`,
`fchdir`, `link`, `symlink`, `unlink`, `readlink`, `fstatat`, `access`,
and `faccessat`.
This also includes limited support for `getcwd()` and `realpath()`.
These return an error if the CWD does not correspond to a Preopen with
an absolute path. They also do not currently expand symlinks.
* os/linux/io_uring: add recvmsg and sendmsg
* Use std.os.iovec and std.os.iovec_const
* Remove msg_ prefix in msghdr and msghdr_const in arm64 etc
* Strip msg_ prefix in msghdr and msghdr_const for linux arm-eabi
* Copy msghdr and msghdr_const from i386 to mips
* Add sockaddr to lib/std/os/linux/mips.zig
* Copy msghdr and msghdr_const from x86_64 to riscv64
* std.BoundedArray: return explicit errors
Makes it easier to mark explicit errors when using BoundedArray
downstream.
* std.BoundedArray.insert() returns Overflow only
Looks like all these functions are at least compiling successfully. I
haven't tried to run their test suites yet.
The one exception is `clone` which is crashing the compiler due to the
inline assembly. Still, this is progress!
`Module.Union.getLayout` now additionally returns a `padding` field
which tells how many bytes are between the final field end offset and
the ending offset of the union. This is used by the LLVM backend to
explicitly insert padding.
LLVM backend: lowering of unions now inserts additional padding so that
LLVM's internals will agree on the ABI size to match what ABI size zig
wants unions to be. This is an alternative to calling LLVMABISizeOfType
and LLVMABIAlignmentOfType which end up crashing when recursive struct
definitions come into play. We no longer ever call these two functions
and the bindings are deleted to avoid future footgun firings.
LLVM backend: lowering of unions now represents untagged unions
consistently. Before it was tripping an assertion.
LLVM backend: switch cases call inttoptr on the case items and condition
if necessary. Prevents tripping an LLVM assertion.
After this commit, we are no longer tripping over any LLVM assertions.
The core of this change is to re-use the escape sequence parsing logic
for parsing both string and character literals.
The actual fix is that UTF-8 encoding was missing for string literals
with \u{...} escape sequences.
* Sema: resolve type fully when emitting an alloc AIR instruction to
avoid tripping assertion for checking struct field alignment.
* LLVM backend: keep a reference to the LLVM target data alive during
lowering so that we can ask LLVM what it thinks the ABI alignment
and size of LLVM types are. We need this in order to lower tuples and
structs so that we can put in extra padding bytes when Zig disagrees
with LLVM about the size or alignment of something.
* LLVM backend: make the LLVM struct type packed that contains the most
aligned union field and the padding. This prevents the struct from
being too big according to LLVM. In the future, we may want to
consider instead emitting unions in a "flat" manner; putting the tag,
most aligned union field, and padding all in the same struct field
space.
* LLVM backend: make structs with 2 or fewer fields return isByRef=false.
This results in more efficient codegen. This required lowering of
bitcast to sometimes store the struct into an alloca, ptrcast, and
then load because LLVM does not allow bitcasting structs.
* enable more passing behavior tests.
Packed structs were tripping an LLVM assertion due to calling
`LLVMConstZExt` from i16 to i16. Solved by using instead
`LLVMConstZExtOrBitCast`.
Unions were tripping an LLVM assertion due to a typo using the union
llvm type to construct an integer value rather than the tag type.
This was introduced in d1a4654834: when a
BufSet clones the keys, it used to assign the new pointers to the old
struct. Fix that by assigning the pointers to the correct, i.e. the new,
struct.
This caused double-free when using arena allocator for the new struct,
also in the test case.
Symbols that have globals used to have their lookup key be the symbol name.
This key is now the offset into the string table.
Imports have both the module name (library name) and name (of the symbol), those strings are now
also being interned. This can save us up to 24bytes per import which have both their module name and name de-duplicated.
Module names are almost entirely the same for all imports, providing us with a big chance of saving us 12 bytes at least.
Just like imports, exports can also have a seperate name than the internal symbol name. Rather than storing the slice,
we now store the offset of this string instead.
For all symbols read from object files as well as generated from Zig code
will now be interned and have their offset into the string table saved on the `Symbol` instead.
Besides interning, local symbols now also use a decl's fully qualified name.
When a decl/symbol is extern/to-be-imported, the name of the decl itself will be used for symbol resolving.
Similarly for symbols that will be exported, will have their 'export name' set.
This is preliminary work for string interning in the wasm linker.
Using an arena would defeat the purpose of de-duplicating strings as we wouldn't be able to free memory
of duplicated strings.
This change also means we can simplify wasm binary parsing, by creating a general purpose parser that
parses the binary into its sections, but untyped. Doing this, allows us to re-use the base of that, for
object file, but also debug info parsing.
* Fix compile error for `zirErrorUnionType`.
* Convert zirMergeErrorSets logic to call `Type.errorSetMerge`.
It does not need to create a Decl as the TODO comment hinted.
* Extract out a function called `resolveInferredErrorSetTy`.
* Rework `resolvePeerTypes` with respect to error unions and
error sets. This is a less complex implementation that passes all the
same tests and uses many fewer lines of code by taking advantage of
the function `coerceInMemoryAllowedErrorSets`.
- Always merge error sets in the order that makes sense, even when
that means `@typeInfo` incompatibility with stage1.
* `Type.errorSetMerge` no longer overallocates.
* Don't skip passing tests.
Similar to how Type.eql was reworked in the previous commit, this commit
reworks Type.hash to check all the different kinds of tags that a Type
can be represented with. It also completes the implementation for all
types except error sets, which need to have Type.eql enhanced as well.
Several issues with pointer types are fixed:
Prior to this commit, Zig would not canonicalize a pointer type with
an explicit alignment to alignment=0 if it matched the pointee ABI
alignment. In order to fix this, `Type.ptr` now takes a Target
parameter. I also moved the host_size canonicalization to `Type.ptr`
since target is now available. Similarly, is_allowzero in the case of
C pointers is now treated as a canonicalization done by the function
rather than a precondition.
in-memory coercion for pointers now properly checks ABI alignment
of pointee types instead of incorrectly treating the 0 value as an
alignment.
Type equality is completely reworked based on the tag() rather than the
zigTypeTag(). It's still semantically based on zigTypeTag() but that
knowledge is implied rather than dictating the control flow of the
logic. Importantly, this fixes cases for opaques, structs, tuples,
enums, and unions, where type equality was incorrectly returning based
on whether the tag() values were equal.
Additionally, pointer type equality now takes into account alignment.
Because we canonicalize non-zero alignment which equals pointee type ABI
alignment to alignment=0, this now can be a simple integer comparison.
Type hashing is implemented for pointers and floats. Array types now
additionally hash their sentinels.
This regressed some behavior tests that were passing but only because
of bugs regarding type equality.
The C backend has a noticeable problem with lowering differently-aligned
pointers (particularly slices) as the same type, causing C compilation
errors due to duplicate declarations.
We were using the array type, not the element type. Also, we should do
the sentinel comparison after we verify that the element types of both
are compatible.
Now we handle -o /dev/null equivalent to -fno-emit-bin because
otherwise our atomic rename into place will fail. This also
makes Zig do less work, avoiding pointless file system operations.
Do the fallible logic in Sema where we have access to error reporting
mechanisms, rather than in Type/Value.
We can't just do the best guess when resolving queries of "is this type
comptime only?" or "what is the ABI alignment of this field?". The
result needs to be accurate. So we need to keep the assertions that the
data is available active, and instead compute the necessary information
before such functions get called.
Unfortunately we are stuck with two versions of such functions because
the various backends need to be able to ask such queries of Types and
Values while assuming the result has already been computed and validated
by Sema.
This also includes two other small fixes:
- Instantiate void TypeInfo fields as void
- Return error in `type.comptimeOnly` on unresolved comptime requirements
This adds a comptime result when comparing a comptime value to an
unsigned integer. For example:
( 0 <= (unsigned runtime value)) => true
(-1 < (unsigned runtime value)) => true
((unsigned runtime value) < -15) => false
This bug only causes a failure on my machine when running
test/behavior/eval.zig directly. If running the full behavior test
suite, std.builtin.TypeInfo will have already resolved its layout,
causing the test to pass.
I'd love to add a test that can reliably reproduce this problem,
but I'm afraid I'm not sure how to reliably create a union with
un-resolved layout.
The ZIR instruction `union_init_ptr` is renamed to `union_init`.
I made it always use by-value semantics for now, not taking the time to
invest in result location semantics, in case we decide to change the
rules for unions. This way is much simpler.
There is a new AIR instruction: union_init. This is for a comptime known
tag, runtime-known field value.
vector_init is renamed to aggregate_init, which solves a TODO comment.
Unless the pointer is a pointer to a function, if the pointee type
has zero-bits, we need to return `MCValue.none` as the `Decl` has
not been lowered to memory, and therefore, any GOT reference will be
wrong.
* fix alignment issues for consts with natural ABI alignment not
matching that of the `ldr` instruction in `aarch64` - solved by
preceeding the `ldr` with an additional `add` instruction to form
the full address before dereferencing the pointer.
* redo selection of segment/section for decls and consts based on
combined type and value
Rather than ping ponging between codegen and the linker to generate the symbols/atoms
for a local constant and its relocations. We now create all neccesary objects within the linker.
This simplifies the code as we can now simply call `lowerUnnamedConst` from anywhere in codegen,
allowing us to further improve lowering constants into .rodata so we do not have to sacrifice
lowering certain types such as decl_ref's where its type is a slice.
* AstGen: remove the setBlockBodyEliding function. This is no longer
needed after 63788b2a51.
* Sema: store_to_block_ptr instruction is handled as
store_to_inferred_ptr or store, as necessary.
Instead of explicitly setting lhs to .none,
check if the lhs instruction was analyzed.
This simpler approach also handles stores from nested blocks correctly.
As of 6249a24, align() is not allowed on packed struct fields
and as such the align(4) was removed from the first field of
EdidOverrideProtocolAttributes. The endgame here is packed struct
backed by explicit integers, in this case a u32, but until that
is implemented put the align(4) on the pointer to avoid breaking
someone's UEFI code in a hard to debug way.
This implements #10113 for the self-hosted compiler only. It removes the
ability to override alignment of packed struct fields, and removes the
ability to put pointers and arrays inside packed structs.
After this commit, nearly all the behavior tests pass for the stage2 llvm
backend that involve packed structs.
I didn't implement the compile errors or compile error tests yet. I'm
waiting until we have stage2 building itself and then I want to rework
the compile error test harness with inspiration from Vexu's arocc test
harness. At that point it should be a much nicer dev experience to work
on compile errors.
* goals
- zig as linker for object files generated by other compilers
- zig-specific runtime features for eventual standardisation
* changes
- missing routines are marked with `missing`
- structure inspired by libgcc docs, but improved order and wording
- rename misspelled functions
- reorder and rephrase compiler_rt.zig to reflect documentation
- potential decimal float or fixed-point arithmetic support:
* 'Decimal float library routines' ca. 120 functions
* 'Fixed-point fractional library routines' ca. 300 functions
thanks to @Vexu for multiple reviews and @scheibo for review
This instruction now just represents loading from a hard-coded adrress
after extracting the other use cases for load_memory into load_got and
load_direct.
Previously, Zig ignored -lgcc_s with a warning that this dependency is
redundant because it is satisfied by compiler-rt. However, sfackler
pointed out that it also provides exception handling functions. So if
Zig sees -lgcc_s on the linker line, it needs to fulfill this dependency
with libunwind.
I also made link_libc inferred to be on if libunwind is linked since
libunwind depends on libc.
Otherwise, we risk collisions in the global symbol table. This is
also an opportunity to generalise and rewrite the symbol table
abstraction.
Also, improve the logs for the symbol table.
Prior to this change, the routine would assume it is called first,
before any symbol was created, thus precluding an option that in the
incremental setting, we might have already pulled a suitably defined
and exported symbol that could collide and/or be replaced by the
symbol synthesised by the linker.
We now correctly implement exporting decls. This means it is possible to export
a decl with a different name than the decl that is doing the export.
This also sets the symbols with the correct flags, so when we emit a relocatable
object file, a linker can correctly resolve symbols and/or export the symbol to the host environment.
This commit also includes fixes to ensure relocations have the correct offset to how other
linkers will expect the offset, rather than what we use internally.
Other linkers accept the offset, relative to the section.
Internally we use an offset relative to the atom.
When generating a relocatable object file, we now emit a custom "reloc.CODE" and "reloc.DATA" section
which will contain the relocations for each section.
Using a new symbol location -> Atom mapping, we can now easily find the corresponding `Atom` from a symbol.
This can be used to construct the symbol table, as well as easier access to a target atom when performing
a relocation for a data symbol.
When creating a relocatable object file, we do no longer perform the following actions:
- Merge data segments
- Calculate stack size
- Relocations
We now also make the stack pointer symbol `undefined` for this use case as well as add the symbol
as an import.
When creating a relocatable object file, emit the symbol table.
We do this by iterating over all atoms, and finding the corresponding
symbols of those. This provides us all the meta information such as size, and offset as well.
This data is required for defined data symbols.
When we emit an object file, the "Names" section does not have to be emitted, as all symbol names
are already in the symbol table, so the names section is redundant.
The mechanism behind initializing a union's tag is a bit complicated,
depending on whether the union is initialized at runtime,
forced comptime, or implicit comptime.
`coerce_result_ptr` now does not force a block to be a runtime context;
instead of adding runtime instructions directly, it forwards analysis to
the respective functions for initializing optionals and error unions.
`validateUnionInit` now has logic to still emit a runtime
`set_union_tag` instruction even if the union pointer is comptime-known,
for the case of a pointer that is not comptime mutable, such as a
variable or the result of `@intToPtr`.
`validateStructInit` looks for a completely different pattern now; it
now handles the possibility of the corresponding AIR instruction for
the `field_ptr` to be missing or the corresponding `store` to be missing.
See the new comment added to the function for more details. An
equivalent change should probably be made to `validateArrayInit`.
`analyzeOptionalPayloadPtr` and `analyzeErrUnionPayloadPtr` functions now
emit a `optional_payload_ptr_set` or `errunion_payload_ptr_set`
instruction respectively if `initializing` is true and the pointer value
is not comptime-mutable.
`storePtr2` now tries the comptime pointer store before checking if the
element type has one possible value because the comptime pointer store
can have side effects of setting a union tag, setting an optional payload
non-null, or setting an error union to be non-error.
The LLVM backend `lowerParentPtr` function is improved to take into
account the differences in how the LLVM values are lowered depending on
the Zig type. It now handles unions correctly as well as additionally
handling optionals and error unions.
In the LLVM backend, the instructions `optional_payload_ptr_set` and
`errunion_payload_ptr_set` check liveness analysis and only do the side
effects in the case the result of the instruction is unused.
A few wasm and C backend test cases regressed, but they are due to TODOs
in lowering of constants, so this is progress.
In Mach engine we're seeing command line arguments to `zig build-lib`
exceed the 32 KiB limit that Windows imposes, due to the number of
sources and compiler flags we must pass in order to build gpu-dawn.
This change fixes the issue by having `Builder` check if the arguments
to a `zig build-*` command are >30 KiB and, if so, writes the arguments
to a file `zig-cache/args/<SHA2 of args>`. Then the command invocation
merely becomes `zig build-lib @<that file>`.
Fixes#10693Fixeshexops/mach#167
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
This change enables `zig build-lib` and friends to take a response file
of command line arguments, for example:
```sh
zig build-lib @args.rsp
```
Which effectively does the same thing as this in Bash:
```sh
zig build-lib $(cat args.rsp)
```
Being able to use a file for arguments is important as one can quickly
exceed the 32 KiB limit that Windows imposes on arguments to a process.
Helps #10693
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
This change refactors the `zig` argument handling (for `build-lib`, etc.
commands) to use a `process.argsWithAllocator` iterator instead of
directly accessing arguments via array indices. This supports the next
commit which will enable us to use a response file argument iterator
here seamlessly.
Helps #10693
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
Windows does Unicode-aware case-insensitivity comparisons for environment variable names. Before, os.getenvW was only doing ASCII case-insensitivity. We can take advantage of RtlEqualUnicodeString in NtDll to get the proper Unicode case insensitivity.
Prior to this commit, the AIR arg instruction kept a reference to a ZIR
string index for the corresponding parameter name. This is used by DWARF
emitting code. However, this is a design flaw because we want AIR
objects to be independent from ZIR.
This commit saves the parameter names into memory managed by
`Module.Fn`. This is sub-optimal because we should be able to get the
parameter names from the ZIR for a function without having them
redundantly stored along with `Fn` memory. However the current way that
ZIR param instructions are encoded does not support this case. They
appear in the same ZIR body as the function instruction, just before it.
Instead, they should be embedded within the function instruction, which
will allow this TODO to be solved. That improvement is too big for this
commit, however.
After this there is one last dependency to untangle, which is for inline
assembly. The issue for that is #10784.
We return compare flags rather than a register which than wrongly
cheats the regalloc into thinking we carry the instruction in the
register which we do not.
If the function call is not extern and we are compiling in debug,
pass function params always on stack. This will improve debugging
capabilities since the params will not be volatile and possibly
clobbered by the procedure code.
Finish implementation of `imul_complex`.
using the provided -L directories before checking if we need to integrate
with system library paths. This prevents false positive of invoking
system cc to find native paths when in fact all dependencies are
satisfied by -L (or --search-prefix to zig build).
* build.zig: remove detour through zig0. I'll add it back as an option
later. This means you can build stage1 with a freshly built zig
without detouring through zig0 again.
* remove unused file windows_script.bat
* drone.yml: remove unnecessary steps
* ninja doesn't need a jobs parameter
* build the release version of zig with zig build instead of cmake.
* build stage2 with -Dstatic-llvm -Duse-zig-libcxx
* Test everything on the Linux CI even if we can't run it, because it's
our fastest machine.
* Test stage2 using a build of stage2 (instead of using `-fno-stage1`)
so that compiler-rt is also built with stage2.
* Additionally test running x86_64-macos on the macOS CI, both the LLVM
backend and x86_64 backend.
This reverts commit baead472d7.
Let's go through the proposal process on this one. I want to push back
on this. My position is that, at the very least, a full trace of command
lines of sub-processes should be printed on failure, with the exception
of opt-in flags such as `--prominent-compile-errors`.
If a '(' is found where the continue expression was expected and it is
on the same line as the previous token issue an error about missing
colon before the continue expression.
- Correctly get discard symbol by first checking if it was discarded or not.
- Remove imports if extern symbols were resolved by an object file.
- Correctly relocate data symbols by ensuring the atom is from the correct file.
- Fix the `Names` section by using the resolved symbols, rather than the ones defined in Zig code.
We now correctly allocate and create atoms for symbols from other object files.
Imports are now also resolved and appended when required.
Besides those changes, we now duplicate all symbol names, so we can correctly
generate unique names for unnamed constants.
TODO: String interning
This implements the merging of all sections, to generate a valid wasm binary where all symbols
have been resolved and their respective sections have been merged into the final binary.
This upstreams the object file parsing from zwld, bringing us closer to being able
to link stage2 code with object files/C-code as well as replacing lld with the self-hosted
linker once feature complete.
Address https://github.com/ziglang/zig/issues/3477
This provides a mechanism for builds to fully report an error to the user and prevent zig from piling on extra noise.
Non-Arm CPUs use u16 as the parameter to __extendhfxf2 and the return value of
__truncxfhf2, so insert appropriate bitcasts in gen_soft_f80_widen_or_shorten.
Otherwise, LLVM might crash because the functions are called in a different way
than its compiler-rt definition.
This fixes stage1 build on SPARCv9, and possibly other non-x86, non-Arm CPUs.
For some errors if the found token is not on the same line as
the previous token, point to the end of the previous token.
This usually results in more helpful errors.
Instead of assuming that the zig version used with `zig build` is
appropriate for building the self-hosted compiler component, we follow
the same path as the cmake build script and build zig0, using that to
produce zig1.o, re-linking to produce stage1.
This allows an arbitrarily old Zig version and corresponding build.zig
script to build all future versions of Zig from source via the bootstrap
compiler. In other words, it allows us to use `zig build` as an
alternative to cmake when bootstrapping.
This updates the C backend to use proper array types.
In order to do that, this commit also:
- fixes up elem_ptr and field_ptr handling
- adds `renderTypecast` (renders in C typecast format, e.g. "int* [10]")
- adds a bit special handling for undefined pointers, which is necessary
to support slice/elem_ptr to undefined decls
Now, the abstracted stack offsets grow in the same direction as
the real stack values in hardware, and allocating stack memory is done
by the taking the last stack offset, adding required abi size
and aligning to the required abi align. Stack handling is now more
natural as it aligns itself with how it works in hardware; hence
stepping through the debugger and printing out different stack
values is intuitive. Finally, the stack pointers are now correctly
aligned to the required (and not necessarily natural) alignment.
std.fmt.parseHexFloat allow both 0x and 0X as prefix, so in order to
keep things consistent. This commit modifies std.fmt.parseWithSign to
check for the prefix case insensitively in order to allow both upper
case and lower case prefix.
This change now allows: 0O, 0B and 0X as prefixes for parsing.
* move DWARF in file if LINKEDIT spilled in dSYM
* update VM addresses for all segments
* introduce copyFileRangeOverlappedAll instead of File.copyRangeAll
since we make lots of overlapping writes in MachO linker
This "feature" of gcc/clang means to treat this as a positional
link object, but using the library search directories as a prefix.
We solve this problem in the CLI layer, using a separate map for
the data since it is an uncommon case.
Closes#10851
On Windows, `argv` is not populated by start code, and instead left as undefined. This is problematic, and can lead to incorrect programs compiling, but panicking when trying to access `argv`. This change causes these programs to produce a compile error on Windows instead, which is far preferable to a runtime panic.
* update number of type abbrevs to match Elf linker
* update `DebugSymbols` to write symbol and string tables
at the end to match the `MachO` linker
* TODO: update segment vm addresses when growing segments in
the binary
* TODO: store DWARF relocations in linker's interned arena
Previously, `suffix` was copied to `output_buffer` at position
`max_end`, thereby writing into reserved space after `max_end`.
This only worked because `suffix` was not larger than
`bytes_needed_for_esc_codes_at_end` (otherwise there'd be a potential
buffer overrun) and no escape codes at end are actually written.
Since 2d5b2bf1c9, escape codes are no
longer written to the end of the buffer. They are now written
exclusively to the front of the buffer.
This allows removing `bytes_needed_for_esc_codes_at_end` and
simplifying the suffix printing logic.
This also fixes the bug that the ellipse suffix was not printed in
Windows terminals because `end.* > max_end` was never true.
In #10859 I moved the `test_node.end()` call after everything else has
been logged. Now the `test_fn.name` is printed by `Progress` itself,
making the additional log obsolete.
The information whether a register is allocated to an instruction is
already encoded in the free_registers "bitmap". Duplicating that
information in the registers map is unnecessary and may lead to
performance degradations.
Implements a cross-platform metadata API, aiming to reduce unnecessary Unix-dependence of the `std.fs` api. Presently, all OSes beside Windows are treated as Unix; this is likely the best way to treat things by default, instead of explicitly listing each Unix-like OS.
Platform-specific operations are not provided by `File.Metadata`, and instead are to be accessed from `File.Metadata.inner`.
Adds:
- File.setPermissions() : Sets permission of a file according to a `Permissions` struct (not available on WASI)
- File.Permissions : A cross-platform representation of file permissions
- Permissions.readOnly() : Returns whether the file is read-only
- Permissions.setReadOnly() : Sets whether the file is read-only
- Permissions.unixSet() : Sets permissions for a class (UNIX-only)
- Permissions.unixGet() : Checks a permission for a class (UNIX-only)
- Permissions.unixNew() : Returns a new Permissions struct to represent the passed mode (UNIX-only)
- File.Metadata : A cross-platform representation of file metadata
- Metadata.size() : Returns the size of a file
- Metadata.permissions() : Returns a `Permissions` struct, representing permissions on the file
- Metadata.kind() : Returns the `Kind` of the file
- Metadata.accessed() : Returns the time the file was last accessed
- Metadata.modified() : Returns the time the file was last modified
- Metadata.created() : Returns the time the file was created (this is an optional, as the underlying filesystem, or OS may not support this)
Methods of `File.Metadata` are also available for the below, so I won't repeat myself
The below may be used for platform-specific functionality
- File.MetadataUnix : The internal implementation of `File.Metadata` on Unices
- File.MetadataLinux : The internal implementation of `File.Metadata` on Linux
- File.MetadataWindows : The implementation of `File.Metadata` on Windows
Big-int functions were updated to respect the provided abi_size, rather
than inferring a potentially incorrect abi_size implicitly.
In combination with the convention that any required padding bits are
added on the MSB end, this means that exotic integers can potentially
have a well-defined memory layout.
Previously the progress displayed the first item as [0/x]. This was
misleading when x is the number of items. The first item should be
displayed as [1/x]
Due to differences in where the output gets emitted in stage1 and stage2,
we were putting the symlink next to the binary rather than in `zig-cache`
directory when building with stage2.
LLVM doesn't support lowering union values, so we have to use unnamed
structs to do it, which means any type that contains a union as an
element, even if it is nested in another type, has to have a mechanism
to detect when it can't be lowered normally and has to resort itself to
an unnamed struct.
This includes arrays.
Comment reproduced here:
Note the following u64 alignments:
x86-linux: 4
x86-windows: 8
LLVM makes x86_fp80 have the following alignment and sizes regardless
of operating system:
x86_64: size=16, align=16
x86: size=12, align=4
However in Zig we override x86-windows to have size=16, align=16
in order for the property to hold that u80 and f80 have the same ABI size.
Fixes "error: destination type 'f80' has size 12 but source type 'u80'
has size 16" when trying to bitcast between f80 and u80 on i386-windows.
Get rid of `std.math.F80Repr`. Instead of trying to match the memory
layout of f80, we treat it as a value, same as the other floating point
types. The functions `make_f80` and `break_f80` are introduced to
compose an f80 value out of its parts, and the inverse operation.
stage2 LLVM backend: fix pointer to zero length array tripping LLVM
assertion. It now checks for when the element type is a zero-bit type
and lowers such thing the same way that pointers to other zero-bit types
are lowered.
Both stage1 and stage2 LLVM backends are adjusted so that f80 is lowered
as x86_fp80 on x86_64 and i386 architectures, and identical to a u80 on
others. LLVM constants are lowered in a less hacky way now that #10860
is fixed, by using the expression `(exp << 64) | fraction` using llvm
constants.
Sema is improved to handle c_longdouble by recursively handling it
correctly for whatever the float bit width is. In both stage1 and
stage2.
* F80Repr extern struct needs no explicit padding; let's match the
target padding.
* stage2: fix lowering of f80 constants.
* stage1: decide ABI size and alignment of f80 based on alignment of
u64. x86 has alignof u64 equal to 4 but arm has it as 8.
* stage2: fix Value.floatReadFromMemory to use F80Repr
replaceAllUsesWith requires the type to be unchanged. So we bitcast
the new global to the old type and use that as the thing to replace
old uses.
Fixes an LLVM assertion found while troubleshooting #10837.
Match changes required to `Elf` linker, which enable lowering
of const slices on `MachO` targets.
Expand `Mir` instructions requiring the knowledge of the containing
atom - pass the symbol index into the linker's table from codegen
via mir to emitter, to then utilise it in the linker.
In `getDeclVAddr`, it may happen that the target `Decl` has not
been allocated space in virtual memory. In this case, we store a
relocation in the linker-global table which we will iterate over
when flushing the module, and fill in any missing address in the
final binary. Note that for optimisation, if the address was resolved
at the time of a call to `getDeclVAddr`, we skip relocating this
atom.
This commit also adds the glue code for lowering const slices in
the ARM backend.
This moves the single bugs behavior tests to the outer branch and disables the test cases
for all non-passing backends.
For the larger files, we move it up a single branch and disable it for the c backend.
All test cases that do pass for the c backend however, are enabled.
This implements the `field_ptr` value for pointers. As the value only provides us with the index,
we must calculate the offset from the container type using said index. (i.e. the offset from a struct field at index 2).
Besides this, small miscellaneous fixes/updates were done to get remaining behavior tests passing:
- We start the function table index at 1, so unresolved function pointers don't can be null-checked properly.
- Implement genTypedValue for floats up to f64.
- Fix zero-sized arguments by only creating `args` for non-zero-sized types.
- lowerConstant now works for all decl_ref's.
- lowerConstant properly lowers optional pointers, so `null` pointers are lowered to `0`.
LLVM union globals have to be lowered as unnamed structs if the
non-most-aligned field is the active tag. In this case it bubbles up so
that structs containing unions have the same restriction.
This fix needs to be applied to optionals and other callsites of
createNamedStruct.
The bug fixed in this commit was revealed in searching for
the cause of #10837.
We need to pad out the file to the required maximum size equal the
final section's offset plus the section's size. We only need to
this when populating initial metadata and only when section header
was updated.
In the previous commit I got mixed up and cut-pasted instead of
copy-pasting. In this commit I made c_stage1.zig additionally included
for stage1 and everything else included for both. So moving forward we
move stuff over from c_stage1.zig to c.zig instead of copying.
For some projects, they can't help themselves, -lgcc_s ends up on the
compiler command line even though it does not belong there. In Zig we
know what -lgcc_s does. It's an alternative to compiler-rt. With this
commit we emit a warning telling that it is unnecessary to put such
thing on the command line, and happily ignore it, since we will fulfill
the dependency with compiler-rt.
When Sema sees a store_node instruction, it now checks for
the possibility of this pattern:
%a = ret_ptr
%b = store(%a, %c)
Where %c is an error union. In such case we need to add to the
current function's inferred error set, if any.
Coercion from error union to error union will be handled ideally if the
operand is comptime known. In such case it does the appropriate
unwrapping, then wraps again.
In the future, coercion from error union to error union should do the
same thing for a runtime value; emitting a runtime branch to check if
the value is an error or not.
`Value.arrayLen` for structs returns the number of fields. This is so
that Liveness can use it for the `vector_init` instruction (soon to be
renamed to `aggregate_init`).
`const` declarations inside comptime blocks were not getting properly
evaluated at compile-time. To accomplish this there is a new ZIR
instruction, `alloc_inferred_comptime`. Actually we already had one
named that, but it got renamed to `alloc_inferred_comptime_mut` to match
the naming convention with the other similar instructions.
Rather than using runtime to perform pointer arithmetic to set the stack offset as
a pointer into a local, we now store the offset as a WValue from the bottom of the stack.
This has the benefit of less instructions, few locals, and less performance impact when
we allocate a value on the virtual stack.
* pass more x64 behavior tests
* return with a TODO error when lowering a decl with no runtime bits
* insert some debug logs for tracing recursive descent down the
type-value tree when lowering types
* print `Decl`'s name when print debugging `decl_ref` value
This updates the test runner for stage2 to emit to stdout with the passed, skipped and failed tests
similar to the LLVM backend.
Another change to this is the start function, as it's now more in line with stage1's.
The stage2 test infrastructure for wasm/wasi has been updated to reflect this as well.
- approach by Hacker's Delight with wrapping subtraction
- performance expected to be similar to addo
- tests with all relevant combinations of min,max with -1,0,+1 and all
combinations of sequences +-1,2,4..,max
* pass air_tag instead of zir_tag
* also pass eval function so that the branch only happens once and the
body of zirUnaryMath is simplified
* Value.sqrt: update to handle f80 and f128 in the normalized way that
includes handling c_longdouble.
Semi-related change: fix incorrect sqrt builtin name for f80 in stage1.
Support for f128, comptime_float, and c_longdouble require improvements
to compiler_rt and will implemented in a later PR. Some of the code in
this commit could be made more generic, for instance `llvm.airSqrt`
could probably be `llvm.airUnaryMath`, but let's cross that
bridge when we get to it.
For large ranges, this is faster than having the caller call setValue() for
each index in the range. Masks wholly covered by the range can be set to
the new mask value in one go, and the two masks at either end that are
partially covered can each set the covered range of bits in one go.
- approach by Hacker's Delight with wrapping addition
- ca. 1.10x perf over the standard approach on my laptop
- tests with all combinations of min,max with -1,0,+1 and combinations of
sequences +-1,2,4..,max
* link: add a virtual function `lowerUnnamedConsts`, similar to
`updateFunc` or `updateDecl` which needs to be implemented by the
linker backend in order to be used with the `CodeGen` code
* elf: implement `lowerUnnamedConsts` specialization where we
lower unnamed constants to `.rodata` section. We keep track of the
atoms encompassing the lowered unnamed consts in a global table
indexed by parent `Decl`. When the `Decl` is updated or destroyed,
we clear the unnamed consts referenced within the `Decl`.
* macho: implement `lowerUnnamedConsts` specialization where we
lower unnamed constants to `__TEXT,__const` section. We keep track of the
atoms encompassing the lowered unnamed consts in a global table
indexed by parent `Decl`. When the `Decl` is updated or destroyed,
we clear the unnamed consts referenced within the `Decl`.
* x64: change `MCValue.linker_sym_index` into two `MCValue`s: `.got_load` and
`.direct_load`. The former signifies to the emitter that it should
emit a GOT load relocation, while the latter that it should emit
a direct load (`SIGNED`) relocation.
* x64: lower `struct` instantiations
In accordance with the requesting issue (#10750):
- `zig test` skips any tests that it cannot spawn, returning success
- `zig run` and `zig build` exit with failure, reporting the command the cannot be run
- `zig clang`, `zig ar`, etc. already punt directly to the appropriate clang/lld main(), even before this change
- Native `libc` Detection is not supported
Additionally, `exec()` and related Builder functions error at run-time, reporting the command that cannot be run
Currently Zig lowers `@intToFloat` for f80 incorrectly on non-x86
targets:
```
broken LLVM module found:
UIToFP result must be FP or FP vector
%62 = uitofp i64 %61 to i128
SIToFP result must be FP or FP vector
%66 = sitofp i64 %65 to i128
```
This happens because on such targets, we use i128 instead of x86_fp80 in
order to avoid "LLVM ERROR: Cannot select". `@intToFloat` must be
lowered differently to account for this difference as well.
We're going to remove the first parameter from this function in the
future. Stage2 already ignores the first parameter. So we put an `@as`
in here to make it work for both.
This adds a new path which avoids using compiler_rt generated div
udivmod instructions in the case that a divisor is less than half the
max usize value. Two half-limb divisions are performed instead which
ensures that non-emulated division instructions are actually used. This
does not improve the udivmod code which should still be reviewed
independently of this issue.
Notably this improves the performance of the toString implementation of
non-power-of-two bases considerably.
Division performance is improved ~1000% based on some coarse testing.
The following test code is used to provide a rough comparison between
the old vs. new method.
```
const std = @import("std");
const Managed = std.math.big.int.Managed;
const allocator = std.heap.c_allocator;
fn fib(a: *Managed, n: usize) !void {
var b = try Managed.initSet(allocator, 1);
defer b.deinit();
var c = try Managed.init(allocator);
defer c.deinit();
var i: usize = 0;
while (i < n) : (i += 1) {
try c.add(a.toConst(), b.toConst());
a.swap(&b);
b.swap(&c);
}
}
pub fn main() !void {
var a = try Managed.initSet(allocator, 0);
defer a.deinit();
try fib(&a, 1_000_000);
// Note: Next two lines (and printed digit count) omitted on no-print version.
const as = try a.toString(allocator, 10, .lower);
defer allocator.free(as);
std.debug.print("fib: digit count: {}, limb count: {}\n", .{ as.len, a.limbs.len });
}
```
```
==> time.no-print <==
limb count: 10849
________________________________________________________
Executed in 10.60 secs fish external
usr time 10.44 secs 0.00 millis 10.44 secs
sys time 0.02 secs 1.12 millis 0.02 secs
==> time.old <==
fib: digit count: 208988, limb count: 10849
________________________________________________________
Executed in 22.78 secs fish external
usr time 22.43 secs 1.01 millis 22.43 secs
sys time 0.03 secs 0.13 millis 0.03 secs
==> time.optimized <==
fib: digit count: 208988, limb count: 10849
________________________________________________________
Executed in 11.59 secs fish external
usr time 11.56 secs 1.03 millis 11.56 secs
sys time 0.03 secs 0.12 millis 0.03 secs
```
Perf data for non-optimized and optimized, verifying no udivmod is
generated by the new code.
```
$ perf report -i perf.data.old --stdio
- Total Lost Samples: 0
-
- Samples: 90K of event 'cycles:u'
- Event count (approx.): 71603695208
-
- Overhead Command Shared Object Symbol
- ........ ....... ................ ...........................................
-
52.97% t t [.] compiler_rt.udivmod.udivmod
45.97% t t [.] std.math.big.int.Mutable.addCarry
0.83% t t [.] main
0.08% t libc-2.33.so [.] __memmove_avx_unaligned_erms
0.08% t t [.] __udivti3
0.03% t [unknown] [k] 0xffffffff9a0010a7
0.02% t t [.] std.math.big.int.Managed.ensureCapacity
0.01% t libc-2.33.so [.] _int_malloc
0.00% t libc-2.33.so [.] __malloc_usable_size
0.00% t libc-2.33.so [.] _int_free
0.00% t t [.] 0x0000000000004a80
0.00% t t [.] std.heap.CAllocator.resize
0.00% t libc-2.33.so [.] _mid_memalign
0.00% t libc-2.33.so [.] sysmalloc
0.00% t libc-2.33.so [.] __posix_memalign
0.00% t t [.] std.heap.CAllocator.alloc
0.00% t ld-2.33.so [.] do_lookup_x
$ perf report -i perf.data.optimized --stdio
- Total Lost Samples: 0
-
- Samples: 46K of event 'cycles:u'
- Event count (approx.): 36790112336
-
- Overhead Command Shared Object Symbol
- ........ ....... ................ ...........................................
-
79.98% t t [.] std.math.big.int.Mutable.addCarry
15.14% t t [.] main
4.58% t t [.] std.math.big.int.Managed.ensureCapacity
0.21% t libc-2.33.so [.] __memmove_avx_unaligned_erms
0.05% t [unknown] [k] 0xffffffff9a0010a7
0.02% t libc-2.33.so [.] _int_malloc
0.01% t t [.] std.heap.CAllocator.alloc
0.01% t libc-2.33.so [.] __malloc_usable_size
0.00% t libc-2.33.so [.] systrim.constprop.0
0.00% t libc-2.33.so [.] _mid_memalign
0.00% t t [.] 0x0000000000000c7d
0.00% t libc-2.33.so [.] malloc
0.00% t ld-2.33.so [.] check_match
```
Closes#10630.
AstGen: Fixed bug where f80 types in source were triggering illegal
behavior.
Value: handle f80 in floating point arithmetic functions.
Value: implement floatRem and floatMod
This commit introduces dependencies on compiler-rt that are not
implemented. Those are a prerequisite to merging this branch.
When setting the break value in an if expression we must explicitly
check if a result location type coercion that needs to happen. This was
already done for switch expression, so let's just imitate that check
and fix for if expressions. To make this possible, we now also propagate
`rl_ty_inst` to sub scopes.
This was found on a user's machine when calling "git" as a child process from msys. Instead of getting BROKEN_PIPE on GetOverlappedREsult, it would occur on ReadFile which would then cause the function to hang because the async operation was never started.
For example, a situation like this is allowed
```zig
extern "c" var stderrp: c_int;
```
In this case, `Module.Var` wrapping `stderrp` will have `lib_name`
populated with the library name where this import is expected.
`ExternFn` will contain a maybe-lib-name if it was defined with
the `extern` keyword like so
```zig
extern "c" fn write(usize, usize, usize) usize;
```
`lib_name` will live as long as `ExternFn` decl does.
These options were removed in 5e63baae8 (CLI: remove --verbose-ast and
--verbose-tokenize, 2021-06-09) but some remainders were left in.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
For PIE targets, we defer getting an address of value until the linker
has allocated all atoms and performed the relocations. In codegen,
we represent this via `MCValue.linker_sym_index` value.
- use usize to decide if register size is big enough to store
multiplication result or if division is necessary
- multiplication routine with check of integer bounds
- wrapping multipliation and division routine from Hacker's Delight
fieldVal handles pointer to pointer to array. This can happen for
example, if a pointer to an array is used as the condition expression of
a for loop.
resolveStructFully handles tuples (by doing nothing).
fixed Type comparison for tuples to handle comptime fields properly.
* resolve_inferred_alloc now gives a proper mutability attribute to the
corresponding alloc instruction. Previously, it would fail to mark
things const.
* slicing: fix the detection for when the end index equals the length
of the underlying object. Previously it was using `end - start` but
it should just use the end index directly. It also takes into account
when slicing a comptime-known slice.
* `Type.sentinel`: fix not handling all slice tags
* zig_clang is fully updated
* zig_llvm is fully updated
Some initial work on codegen.cpp is in place for upgrading to LLVM's
new opaque pointers. However there is much more to be done.
A few of zig llvm bindings for deprecated functions have been updated;
more need to be updated.
Singular tests (such as in the bug ones) are moved to top level with exclusions for non-passing backends.
The big behavior tests such as array_llvm and slice are moved to the inner scope with the C backend disabled.
They all pass for the wasm backend now
We now calculate the total stack size required for the current frame.
The default alignment of the stack is 16 bytes, and will be overwritten when the alignment
of a given type is larger than that.
After we have generated all instructions for the body, we calculate the total stack size
by forward aligning the stack size while accounting for the max alignment.
We then insert a prologue into the body, where we substract this size from the stack pointer
and save it inside a bottom stackframe local. We use this local then, to calculate
the stack pointer locals of all variables we allocate into the stack.
In a future iteration we can improve this further by storing the offsets as a new `stack_offset` `WValue`.
This has the benefit of not having to spend runtime cost of storing those offsets, but instead we append
those offsets whenever we need the value that lives in the stack.
Implements the instruction `vector_init` for structs and arrays.
For arrays, it checks if the element must be passed by reference or not.
When not, it can simply use the `offset` field of a store instruction to copy the values
into the array. When it is byref, it will move the pointer by the element size, and then perform
a store operation. This ensures types like structs will be moved into the right position.
For structs we will always move the pointer, as we currently cannot verify if all fields are
not by ref.
This implements lowering elem_ptr for decl's and constants.
To generate the correct pointer, we perform a relocation by using the addend
that represents the offset. The offset is calculated by taking the element's size
and multiplying that by the index.
For constants this generates a single immediate instruction, and for decl's
this generates a single pointer address.
* pad out (non-packed) struct fields when lowering to bytes to be
saved in the binary - prior to this change, fields would be
saved at non-aligned addresses leading to wrong accesses
* add a matching test case to `behavior/struct.zig` tests
* fix offset to field calculation in `struct_field_ptr` on `x86_64`
Clarify that `astgen.advanceSourceCursor` already increments absolute
values of the line and columns numbers; i.e., `GenZir.calcLine` is thus
not only obsolete but wrong by design.
Incidentally, this clean up allows for specifying the `FnDecl` line
numbers for DWARF use correctly as relative values with respect to
the start of the parent `Decl`. This `Decl` in turn has its line number
information specified relatively to its parent `Decl`, and so on, until
we reach the global scope.
This commit fixes two related things:
1. If the loop goes all the way through the slice without a match, on
the last iteration `mid == symbols.len - 1` which causes
`&symbols[mid + 1]` to be out of bounds. End one step before that
instead.
2. If the address we're looking for is greater than the address of the
last symbol in the slice, we now match it to that symbol. Previously,
we would miss this case since we only matched if the address was _in
between_ the address of two symbols.
which is the index of the key that already exists in the hash map.
This enables the use case of using `AutoArrayHashMap(void, void)` which
may seem surprising at first, but is actually pretty handy!
This commit includes a proof-of-concept of how I want to use it, with a
new InternArena abstraction for stage2 that provides a compact way to
store values (and types) in an "internment arena", thus making types
stored exactly once (per arena), representable with a single u32 as a
reference to a type within an InternArena, and comparable with a
simple u32 integer comparison. If both types are in the same
InternArena, you can check if they are equal by seeing if their index is
the same.
What's neat about `AutoArrayHashMap(void, void)` is that it allows us to
look up the indexes by key, *without actually storing the keys*.
Instead, keys are treated as ephemeral values that are constructed as
needed.
As a result, we have an extremely efficient encoding of types and
values, represented only by three arrays, which has no pointers, and can
therefore be serialized and deserialized by a single writev/readv call.
The `map` field is denormalized data and can be computed from the other
two fields.
This is in contrast to our current Type/Value system which makes
extensive use of pointers.
The test at the bottom of InternArena.zig passes in this commit.
I hit the "quotes in an RSP file" issue when trying to compile gRPC using
"zig cc". As a fun exercise, I decided to see if I could fix it myself.
I'm fully open to this code being flat-out rejected. Or I can take feedback
to fix it up.
This modifies (and renames) _ArgIteratorWindows_ in process.zig such that
it works with arbitrary strings (or the contents of an RSP file).
In main.zig, this new _ArgIteratorGeneral_ is used to address the "TODO"
listed in _ClangArgIterator_.
This change closes#4833.
**Pros:**
- It has the nice attribute of handling "RSP file" arguments in the same way it
handles "cmd_line" arguments.
- High Performance, minimal allocations
- Fixed bug in previous _ArgIteratorWindows_, where final trailing backslashes
in a command line were entirely dropped
- Added a test case for the above bug
- Harmonized the _ArgIteratorXxxx._initWithAllocator()_ and _next()_ interface
across Windows/Posix/Wasi (Moved Windows errors to _initWithAllocator()_
rather than _next()_)
- Likely perf benefit on Windows by doing _utf16leToUtf8AllocZ()_ only once
for the entire cmd_line
**Cons:**
- Breaking Change in std library on Windows: Call
_ArgIterator.initWithAllocator()_ instead of _ArgIterator.init()_
- PhaseMage is new with contributions to Zig, might need a lot of hand-holding
- PhaseMage is a Windows person, non-Windows stuff will need to be double-checked
**Testing Done:**
- Wrote a few new test cases in process.zig
- zig.exe build test -Dskip-release (no new failures seen)
- zig cc now builds gRPC without error
fixes https://github.com/ziglang/zig/issues/10719
compiler_rt already provides __muloti4 but libc++ is also providing it and when linking libc++ it causes a crash on my windows x86_64 machine.
* comptime known 0 as a numerator returns comptime 0 independent of
denominator.
* negative numerator and denominator are allowed when the remainder is
zero because that means the modulus would be also zero.
* organize math behavior tests
The buffer `buf` contains N (= `slice_sizes.len`) slices followed by the
N null-terminated arguments. The N null-terminated arguments are stored
in the `contents` array list. Thus, `buf` size should be:
@sizeOf([]u8) * slice_sizes.len + contents_slice.len
Instead of:
@sizeOf([]u8) * slice_sizes.len + contents_slice.len + slice_sizes.len
This bug was found thanks to the gpa allocator which checks if freed
size matches allocated sizes for large allocations.
stage1: fix issue with to_twos_complement that caused a compile error when performing wrapping addition on two signed ints, both of which have the minimum possible value
I found that after switching from my custom Guid parser to the one in std that it increased zigwin32 build times substantially (from 40 seconds to over 10 minutes). More information can be found in the benchmark PR I created here: https://github.com/ziglang/gotta-go-fast/pull/21 . This PR ports my GUID parser to std so all projects can leverage the faster comptime performance.
This commit fixes an out of bounds write that can occur when
formatting certain float values. The write messes up the stack and
causes incorrect results, segfaults, or nothing at all, depending on the
optimization mode used.
The `errol` function writes the digits of the float into `buffer`
starting from index 1, leaving index 0 untouched, and returns `buffer[1..]`
and the exponent. This is because `roundToPrecision` relies on index 0 being
unused in case the rounding adds a digit (e.g rounding 999.99
to 1000.00). When this happens, pointer arithmetic is used
[here](0e6d2184ca/lib/std/fmt/errol.zig (L61-L65))
to access index 0 and put the ones digit in the right place.
However, `errol3u` contains two special cases: `errolInt` and `errolFixed`,
which return from the function early. For these two special cases
index 0 was never reserved, and the return value contains `buffer`
instead of `buffer[1..]`. This causes the pointer arithmetic in
`roundToPrecision` to write out of bounds, which in the case of
`std.fmt.formatFloatDecimal` messes up the stack and causes undefined behavior.
The fix is to move the slicing of `buffer` to `buffer[1..]` from `errol3u`
to `errol` so that both the default and the special cases operate on the sliced
buffer.
This library contains:
- The global constants as used by C code.
- An Insn struct that implements can generate all the BPF instructions.
- A simple BPF virtual machine implementation that can be used for
testing programs. This has complete code-coverage and has been
extensively fuzzed.
This is a patch to glibc features.h which makes
_DYNAMIC_STACK_SIZE_SOURCE undefined unless the version is >= 2.34.
This feature was introduced with glibc 2.34 and without this patch, code
built against these headers but then run on an older glibc will end up
making a call to sysconf() that returns -1 for the value of SIGSTKSZ
and MINSIGSTKSZ.
Closes#10713
Instead of defaulting to false, just keep the option as optional to
communicate default to the build system.
Fixes one problem with building the compiler for single-threaded
targets.
LLVM bitcast wants integers that match the number of bits. So the const
bitcast has to use an i80, not an i128.
This commit makes the behavior tests fail for me, so it seems I did not
correctly construct the type. But it gets rid of the LLVM segfault.
I noticed that the strategy of memcpy the buf worked if I simply did an
LLVMConstTrunc() on the i128 to make it into an i80 before the
LLVMConstBitCast().
But is that correct in the face of different endianness? I'm not sure.
This fixes the use of multiple `Iterator`s in a row on a directory.
Previously, on many platforms, using an `Iterator` on an
already-iterated directory would give no entries.
Fixing this involved seeking to the beginning of the directory on the
first call of `next()`.
The idea is that this type gains the relevant low-level instruction emitting
functions, and that higher-level checks and deduplications are performed
somewhere else.
The spirv spec generator now also generates some support information:
Opcode gains a function to query a Zig type representing the operands
of the opcode. The idea is that this will enable a richer interface for
emitting spirv instructions.
`ArrayList.ensureTotalCapacityPrecise` uses `Allocator.reallocAtLeast` under
the hood, which can return more than `new_capacity` bytes if `alignment
!= @alignOf(T)`. This implementation of `clone` assures that
the case of `ensureTotalCapacityPrecise` is handled correctly.
Thanks @Vexu and @squeek502 for pointing this out.
First step towards #10634.
Treating stub files as C++ allows to use zig c++ as a host
compiler for nvcc.
Treating cu files as C++ allow using zig c++ as a host compiler in
CMake. CMake calls the host compiler with -E on a cu file to identify
the compiler.
Using zig c++ to directly compile CUDA code is untested.
check the set of passing tests; move towards the disabling logic being
inside each test rather than which files are included.
this enables a few more passing tests.
Takes advantage of the pattern already established with
array_init_anon. Also upgrades array_init (non-anon) to the pattern.
Implements comptime struct value equality and pointer value hashing.
This is only relevant for ELF files.
I also fixed a bug where passing a zig source file to `zig cc` would
incorrectly punt to clang because it thought there were no positional
arguments.
The freeze/unfreeze API replaces the exceptions API for hopefully
preventing bugs in codegen code using the RegisterManager. The
exceptions API is still available for backwards compatibility and will
be removed once all backends transition to the new freeze/unfreeze
API.
Implements slice types including `[]const u8` for passing as
formal parameters in DWARF. Breaking on a function accepting
a slice in `gdb` will now yield the same behavior as stage1 and/or
LLVM backend:
```zig
fn sumArrayLens(a: []const u32, b: []const u8) usize {
return a.len + b.len;
}
```
Both `a` and `b` can now be inspected in the debugger:
```
Breakpoint 1, sumArrayLens (a=..., b=...) at arr.zig:59
(gdb) p a
$1 = {ptr = 0x7fffffff685c, len = 5}
(gdb) p b
$2 = {ptr = 0x7fffffff683d "\252\252\252\\h\377\377\377\177", len = 3}
(gdb)
```
* remove `LoweringError` error set from `Emit.zig` - it actually
was less than helpful; it's better to either not throw an error
since there can be instructions with mismatching operand sizes
such as `movsx` or assert on a by instruction-basis. Currently,
let's just pass through and see how we fare.
* when moving integers into registers, check for signedness and move
with zero- or sign-extension if source operand is smaller than 8
bytes. The destination operand is always assumed to be full-width,
i.e., 8 bytes.
* clean up `airTrunc` a little to match the rest of CodeGen inst
implementations.
Zig calls it aarch64. Linux calls it arm64. Currently lib/libc/include
has both arm64 and aarch64, which is quite confusing.
tools/update-linux-headers.zig was executed against the latest stable
linux patch version, therefore some other minor header updates. I will
update the wiki on how to do it once this PR is accepted.
Before this commit, compiling an empty main with Stage 2 on macOS x86_64 results in
```
../stage2/bin/zig build-exe -ODebug -fLLVM empty_main.zig
error: sub-compilation of compiler_rt failed
[...]/zig/stage2/lib/zig/std/special/compiler_rt/os_version_check.zig:26:10: error: TODO: Sema.zirStructInit for runtime-known struct values
```
By assigning the value to a variable we can sidestep the issue for now.
After #10656, function pointers are represented with e.g.
`*const fn()void` rather than `fn()void`.
This commit adds code to translate-c to emit different code
depending on whether the output zig source code is intended
to be compiled with stage1 or stage2.
Ideally we will have stage1 and stage2 support the exact same
Zig language, but for now they diverge because I would rather
focus on finishing and shipping stage2 than implementing the
features in stage1.
rather than unconditionally prepending double underscore to all
identifiers. Also, use the prefix `zig_e_` instead of `__`. Also, avoid
triggering this escaping when rendering an identifier and there has
already been a prefix printed.
Augment relocation tracking mechanism to de-duplicate potential
creation of base as well as composite types while unrolling
composite types in the linker - there is still potential for
further space optimisation by moving all type information into
a separate section `.debug_types` and providing references to
entries within that section whenever required (e.g., `ref4` form).
Currently, we duplicate type definitions on a per-decl basis.
Anyhow, with this patch, an example function signature of the following
type:
```zig
fn byPtrPtr(ptr_ptr_x: **u32, ptr_x: *u32) void {
ptr_ptr_x.* = ptr_x;
}
```
will generate the following `.debug_info` for formal parameters:
```
<1><1aa>: Abbrev Number: 3 (DW_TAG_subprogram)
<1ab> DW_AT_low_pc : 0x8000197
<1b3> DW_AT_high_pc : 0x2c
<1b7> DW_AT_name : byPtrPtr
<2><1c0>: Abbrev Number: 7 (DW_TAG_formal_parameter)
<1c1> DW_AT_location : 1 byte block: 55 (DW_OP_reg5 (rdi))
<1c3> DW_AT_type : <0x1df>
<1c7> DW_AT_name : ptr_ptr_x
<2><1d1>: Abbrev Number: 7 (DW_TAG_formal_parameter)
<1d2> DW_AT_location : 1 byte block: 54 (DW_OP_reg4 (rsi))
<1d4> DW_AT_type : <0x1e4>
<1d8> DW_AT_name : ptr_x
<2><1de>: Abbrev Number: 0
<1><1df>: Abbrev Number: 5 (DW_TAG_pointer_type)
<1e0> DW_AT_type : <0x1e4>
<1><1e4>: Abbrev Number: 5 (DW_TAG_pointer_type)
<1e5> DW_AT_type : <0x1e9>
<1><1e9>: Abbrev Number: 4 (DW_TAG_base_type)
<1ea> DW_AT_encoding : 7 (unsigned)
<1eb> DW_AT_byte_size : 4
<1ec> DW_AT_name : u32
```
This makes all union test cases succeed.
`rem` was also implemented as all we had to do is enable the instruction.
Loading and storing values based on ABI-size was simplified to a direct abiSize() call.
We also enabled all the newly passing test cases and disable them for all non-passing backends.
All of those test cases were verified to see if they perhaps already pass for the c-backend.
AstGen:
* rename the known_has_bits flag to known_non_opv to make it better
reflect what it actually means.
* add a known_comptime_only flag.
* make the flags take advantage of identifiers of primitives and the
fact that zig has no shadowing.
* correct the known_non_opv flag for function bodies.
Sema:
* Rename `hasCodeGenBits` to `hasRuntimeBits` to better reflect what it
does.
- This function got a bit more complicated in this commit because of
the duality of function bodies: on one hand they have runtime bits,
but on the other hand they require being comptime known.
* WipAnonDecl now takes a LazySrcDecl parameter and performs the type
resolutions that it needs during finish().
* Implement comptime `@ptrToInt`.
Codegen:
* Improved handling of lowering decl_ref; make it work for
comptime-known ptr-to-int values.
- This same change had to be made many different times; perhaps we
should look into merging the implementations of `genTypedValue`
across x86, arm, aarch64, and riscv.
This commit updates stage2 to enforce the property that the syntax
`fn()void` is a function *body* not a *pointer*. To get a pointer, the
syntax `*const fn()void` is required.
ZIR puts function alignment into the func instruction rather than the
decl because this way it makes it into function types. LLVM backend
respects function alignments.
Struct and Union have methods `fieldSrcLoc` to help look up source
locations of their fields. These trigger full loading, tokenization, and
parsing of source files, so should only be called once it is confirmed
that an error message needs to be printed.
There are some nice new error hints for explaining why a type is
required to be comptime, particularly for structs that contain function
body types.
`Type.requiresComptime` is now moved into Sema because it can fail and
might need to trigger field type resolution. Comptime pointer loading
takes into account types that do not have a well-defined memory layout
and does not try to compute a byte offset for them.
`fn()void` syntax no longer secretly makes a pointer. You get a function
body type, which requires comptime. However a pointer to a function body
can be runtime known (obviously).
Compile errors that report "expected pointer, found ..." are factored
out into convenience functions `checkPtrOperand` and `checkPtrType` and
have a note about function pointers.
Implemented `Value.hash` for functions, enum literals, and undefined values.
stage1 is not updated to this (yet?), so some workarounds and disabled
tests are needed to keep everything working. Should we update stage1 to
these new type semantics? Yes probably because I don't want to add too
much conditional compilation logic in the std lib for the different
backends.
There are some differences vs. the union encoding in the LLVM backend:
- Tagged unions with a 0-bit payload do not become their tag type. Instead,
they are a struct with an empty `union` as their payload field.
- We do not order the `payload`/`tag` storage based on their alignment
An attempt to normalize some of the function names in build.zig. Normalize add*Dir to add*Path. Also use "Library" instead of the "Lib" abbreviation.
The PR does not remove the old names, only adds the new normalized ones to faciliate a transition period.
The size of a GUID is not platform-dependent, it's always a fixed number of bits. So I've updated guid to use fixed bit integer types rather than platform-dependent C integer types.
* AstGen: use Ast.zig helper methods to avoid copy pasting token counting logic
- take advantage of the `first_doc_comment` field we already have for
param AST nodes
* Add missing ZIR docs
There are some restrictions here.
- We either need C11 or a compiler that supports the aligned attribute
- We cannot provide align less than the type's natural C alignment.
Looking at the BufferedWriter assembly generated, one can see that is
has to do a lot of work, just to copy over some bytes and increase an
offset. This is because the LinearFifo is a much more general construct
than what BufferedWriter needs and the optimizer cannot prove that we
don't need to do this extra work.
Replaces the inflate API from `inflateStream(reader: anytype, window_slice: []u8)` to
`decompressor(allocator: mem.Allocator, reader: anytype, dictionary: ?[]const u8)` and
`compressor(allocator: mem.Allocator, writer: anytype, options: CompressorOptions)`
Read bytes to check expected values instead of reading and hashing them.
Hashing is a waste of time when we can just read and compare.
This also removes a dependency on std.crypto.hash.sha2.Sha256 for tests.
If there is a big atom available for re-use in the free list, and
it's the last atom in section, it's ideal capacity might span the
entire section in which case we do not want to calculate the actual
end VM addr of the symbol since it may overflow. Instead, we just take
the max capacity available as end VM addr estimate. In this case,
the max capacity equals `std.math.maxInt(u64)`.
Instead of using `push` and `pop` combo, we now re-use our stack
allocation mechanism which means we don't have to worry about
16-byte stack adjustments on macOS as it is handled automatically
for us. Another benefit is that we don't have to backpatch stack
offsets when pulling args from the stack.
The previous commit that implemented doc comment zir support for
decls did not properly account for all the possible attribute
keyword combinations (threadlocal, extern, and such).
Previously, optional slices returned the pointer size as abi size.
We now account for slices to calculate the correct size which is abi-alignment + slice ABI size.
To generate better code for tuples, we detect a tuple operand in
storePtr, and analyze field loads and stores directly. This avoids
an extra allocation + memcpy which would occur if we used `coerce`.
When asking a struct or union whether the type requires comptime, it may
need to ask itself recursively, for example because of a field which is
a pointer to itself. This commit adds a field to each to keep track of
when computing the "requires comptime" value and returns `false` if the
check is already ongoing.
* AIR instruction vector_init gains the ability to init arrays and
tuples in addition to vectors. This will probably also gain the
ability to initialize structs and be renamed to `aggregate_init`.
* AstGen prefers to use an `anon_array_init` ZIR instruction for
local variables when the init expr is an array literal and there is
no type.
Prior to this change, `__DATA,__bss` and `__DATA,__thread_bss` would
get actually, physically written out to the output file, unnecessarily
filling the output file with 0s.
When a constant will be passed by reference, such as a struct, we will call into genTypedValue
to lower the constant to bytes and store them into the `rodata` section. We will then return the address
of this constant as a `WValue`.
This change means we will have all constants lowered during compilation time, and no longer have
to sacrifice runtime to lower them onto the stack.
This is more like a temp hack than anything else - I think the
mechanism we use for adjusting the stack when pushing args onto
the stack could/should be reused - i.e., we should just calculate
the stack alignment before each call and then reset the `rsp`
rather than relying on the current hack in `gen()` logic.
* push the arguments in reverse order
* add logic for pushing args of any abi size to stack - very similar to
`genSetStack` however, uses `.rsp` as the base register
* increment and decrement `.rsp` if we called a function with args on
the stack in `airCall`
* add logic for recovering args from the caller's stack in the callee
This allows us to get rid of unused fields when generating code for non-function decls.
We can now create seperate instances of `DeclGen` which in turn can then be used
to generate the code for a constant.
Besides those reasons, it will be much easier to switch to the generic purpose `codegen.zig` that any
backend should use. Allowing us to deduplicate this code.
The backend can create annonymous local symbols. This can be used for constants
that will be passed by reference so it will not have to be lowered to the stack, and then
stored into the data section. This also means it's valid to return a pointer to a constant array.
Those local symbols that are created, will be managed by the parent decl. Free'ing the parent decl,
will also free all of its locals.
When a local symbol was created, the index of said symbol will be returned and saved in the `memory`
tag of a `WValue` which is then memoized. This means that each 'emit' of this WValue will create a relocation
for that constant/symbol and the actual pointer value will be set after relocation phase.
Due to the new structure of lowerConstant, we can now simplify the logic in a lot of situations.
- We no longer have to check the `WValue`'s tag to determine how to load/store a value.
- We can now provide simple memcopy's for aggregate types.
- Constants are now memoized, meaning we do no longer lower constants on each callsite.
* rename `entry` to `entry_symbol_name` for the zig build API
* integrate with `zig cc` command line options
* integrate with COFF linking with LLD
* integrate with self-hosted ELF linker
* don't put it in the hash for MachO since it is ignored
In the behavior test listings, I had to move type_info.zig test import
to a section that did not include the x86 backend because it got to the
point where adding another test to the file, even if it was an empty
test that just returned immediately, caused a runtime failure when
executing the test binary.
Anyway, type info for opaques is implemented, and the declarations slice
is shared between it, enums, and unions.
Still TODO is the `data` field of a `Declaration`. I want to consider
removing it from the data returned from `@typeInfo` and introducing
`@declInfo` or similar for this data. This would avoid the complexity of
a lazy mechanism.
* introduce new Mir tag `mov_mem_index_imm` which selects instruction
of the form `OP ptr [reg + rax*1 + imm32], imm32` where the encoded
flags select the appropriate ptr width for memory store operation
(note that scale is fixed and set at 1)
Instead use the standarized option for communicating the
zig compiler backend at comptime, which is `zig_backend`. This was
introduced in commit 1c24ef0d0b.
The ZIR instructions `switch_capture_else` and `switch_capture_ref` are
removed because they are not needed. Instead, the prong index is set to
max int for the special prong.
Else prong with error sets is not handled yet.
Adds a new behavior test because there was not a prior on to cover only
the capture value of else on a switch.
Previously, breaking from an outer block at comptime would result in
incorrect control flow. Now there is a mechanism, `error.ComptimeBreak`,
similar to `error.ComptimeReturn`, to send comptime control flow further
up the stack, to its matching block.
This commit also introduces a new log scope. To use it, pass
`--debug-log sema_zir` and you will see 1 line per ZIR instruction
semantically analyzed. This is useful when you want to understand what
comptime control flow is doing while debugging the compiler.
One more `switch` test case is passing.
* implement `genSetStack` for `ptr_stack_offset`
* handle `ptr_add`
* implement storing from register into pointer in register
* split alignment and array tests into those that pass on x86_64 and
those that do not
* pass more tests on x86_64
* Fix incorrect result when the first digit after the decimal point is not 0-9 - eg 0x0.ap0
* Fix compiler panic when the number starts with `0X` with a capital `X` - eg 0X0p0
* Fix compiler panic when the number has a decimal point immediately after `0x` - eg 0x.0p0
Calling `insert` on a `std.MultiArrayList` currently fails with a compiler error due to using a `try` without the `!` annotation on the return type
```zig
this.list.insert(allocator, 0, listener);
```
```zig
/Users/jarred/Build/zig/lib/std/multi_array_list.zig:192:13: error: expected type 'void', found '@typeInfo(@typeInfo(@TypeOf(std.multi_array_list.MultiArrayList(src.javascript.jsc.node.types.Listener).ensureUnusedCapacity)).Fn.return_type.?).ErrorUnion.error_set'
try self.ensureUnusedCapacity(gpa, 1);
```
Beyond adding default zero-initialization, this commit changes undefined
initialization to zero, as some cases reserved the padding and on other
cases, I've found some systems act strange when giving uninit instead of
zero even when it shouldn't be an issue, one example being
FileProtocol.Open's attributes, which *should* be ignored when not
creating a file, but ended up giving an unrelated error.
There is a mechanism to avoid redundant `as` ZIR instructions which is
to pass `ResultLoc.coerced_ty` instead of `ResultLoc.ty` when it is
known by AstGen that Sema will do the coercion.
This commit downgrades `coerced_ty` to `ty` when a result location
passes through an expression that branches, such as `if`, `switch`,
`while`, and `for`, causing the `as` ZIR instruction to be emitted.
This ensures that the type of a result location will be applied to, e.g.
a `comptime_int` on either side of a branch on a runtime condition.
I've seen having this be wrong break some cross-compilers, and it's
also how it is in other files so it's best to be consistent.
It's also just the actual casing of the file.
It is the job of codegen backends to mark Decls that are referenced as
alive so that the frontend does not sweep them with the garbage. This
commit unifies the code between the backends with an added method on
Decl.
The implementation is more complete than before, switching on the Decl
val tag and recursing into sub-values.
As a result, two more array tests are passing.
also use the common naming convention for glibc versions ("2.33" rather
than "2-33").
I also verified that these files are exactly identical to the previous
files from before zig updated to glibc 2.34.
__libc_start_main() from glibc.2.33.so or older needs to have a __libc_csu_init function callback parameter.
glibc-2.34 on the other hand has a different __libc_start_main() that does not use it,
and the start.S file from glibc-2.34 no longer construct the init function and pass null when calling __libc_start_main.
So, When targetting an older glibc, use the start.s files as they were in glibc-2.33 and construct the __libc_csu_init function.
fixes#10386#10512
Introduce `validate_array_init_comptime`, similar to
`validate_struct_init_comptime` introduced in
713d2a9b38.
`zirValidateArrayInit` is improved to detect comptime array literals and
emit AIR accordingly. This code is very similar to the changes
introduced in that same commit for `zirValidateStructInit`.
The C backend needed some improvements to continue passing the same set
of tests:
* `resolveInst` for arrays now will add a local `static const` with the
array value and so then `elem_val` instructions reference that local.
It memoizes accesses using `value_map`, which is changed to use
`Air.Inst.Ref` as the key rather than `Air.Inst.Index`.
* This required a mechanism for writing to a "header" which is lines
that appear at the beginning of a function body, before everything
else.
* dbg_stmt output comments rather than `#line` directives.
TODO comment reproduced here:
We need to re-evaluate whether to emit these or not. If we naively emit
these directives, the output file will report bogus line numbers because
every newline after the #line directive adds one to the line.
We also don't print the filename yet, so the output is strictly unhelpful.
If we wanted to go this route, we would need to go all the way and not output
newlines until the next dbg_stmt occurs.
Perhaps an additional compilation option is in order?
`Value.elemValue` is improved to support `elem_ptr` values.
Handle `__DATA,.rustc` section containing `rustc` metadata - this
is required to get crates like `serde_derive` link properly.
Note to self: this special section has to be copied __verbatim__
from the relocatable object file - this includes preserving its size
even though unpadded according the section's required alignment.
AIR:
* `array_elem_val` is now allowed to be used with a vector as the array
type.
* New instructions: splat, vector_init
AstGen:
* The splat ZIR instruction uses coerced_ty for the ResultLoc, avoiding
an unnecessary `as` instruction, since the coercion will be performed
in Sema.
* Builtins that accept vectors now ignore the type parameter. Comment
from this commit reproduced here:
The accepted proposal #6835 tells us to remove the type parameter from
these builtins. To stay source-compatible with stage1, we still observe
the parameter here, but we do not encode it into the ZIR. To implement
this proposal in stage2, only AstGen code will need to be changed.
Sema:
* `clz` and `ctz` ZIR instructions are now handled by the same function
which accept AIR tag and comptime eval function pointer to
differentiate.
* `@typeInfo` for vectors is implemented.
* `@splat` is implemented. It takes advantage of `Value.Tag.repeated` 😎
* `elemValue` is implemented for vectors, when the index is a scalar.
Handling a vector index is still TODO.
* Element-wise coercion is implemented for vectors. It could probably
be optimized a bit, but it is at least complete & correct.
* `Type.intInfo` supports vectors, returning int info for the element.
* `Value.ctz` initial implementation. Needs work.
* `Value.eql` is implemented for arrays and vectors.
LLVM backend:
* Implement vector support when lowering `array_elem_val`.
* Implement vector support when lowering `ctz` and `clz`.
* Implement `splat` and `vector_init`.
By placing the stack at the start of the memory section, we prevent the runtime
from silently overwriting the global declarations and instead trap.
We do however, allow users to overwrite this behavior by setting the global-base,
which puts the stack at the end of the memory section and the static data at the base that was specified.
The reason a user would want to do this, is when they are sure the stack will not overflow and they want
to decrease the binary size as the offsets to the static memory are generally smaller.
(Having the stack in front, means that accessing the memory after the stack has a bigger offset when loading/storing from memory).
This allows users to add file paths to device paths, which is often used
in methods like `boot_services.loadImage` and `boot_services.startImage`,
which take a device path with an additional file path appended to locate
the image.
UEFI uses `\` for paths exclusively. This changes std.fs.path to use `\`
for UEFI path joining. Also adds a few tests regarding it, specifically
in making sure double-separators do not result from path joining, as the
UEFI spec says to convert any that result from joining into single
separators (UEFI Spec Version 2.7, pg. 448).
Prior to this change, even if the use specified the sysroot on the
compiler line like so
```
zig cc --sysroot=/path/to/sdk
```
it would only be used as a prefix to include paths and not as a prefix
for `zig ld` linker.
- Add an `Metadata.isFree` helper method.
- Implement `Metadata.isTombstone` and `Metadata.isFree` with `@bitCast` then comparing to a constant. I assume `@bitCast`-then-compare is faster than the old method because it only involves one comparison, and doesn't require bitmasking.
- Summary of benchmarked changes (`gotta-go-fast`, run locally, compared to master):
- 3/4 of the hash map benchmarks used ~10% fewer cycles
- The last one (project Euler) shows 4% fewer cycles.
- This implements the float_to_int AIR instruction.
- Lowering a decl_ref to a slice was previously assumed to contain a pointer
to a slice, rather than an array. This is now fixed, making `@src()` work as well.
- Some preliminary work on 128bit integers have been done to find out what needs to be done
to implement 128bit arithmetic.
* stage2: put decls in different MachO sections
Use `getDeclVAddrWithReloc` when targeting MachO backend rather than
`getDeclVAddr` - this fn returns a zero vaddr and instead creates a
relocation on the linker side which will get automatically updated
whenever the target decl is moved in memory. This fn also records
a rebase of the target pointer so that its value is correctly slid
in presence of ASLR.
This commit enables `zig test` on x86_64-macos.
* stage2: fix output section selection for type,val pairs
It is possible for Zig to emit field ptr instructions to fields whos
type is zero sized. In this case llvm should return a pointer which
points to the next none zero sized parameter.
`runtime_param_index` is used to get the parameter type from `fn_type`,
but this variable was not incremented for zero sized parameters, causing
two zero sized parameters of different type to cause miss complication.
resolveTypeForCodegen is called when we needed to resolve a type fully,
even through pointer. This commit fully implements this, even through
pointer fields on structs and unions.
The function has now also been renamed to resolveTypeFully
This exposes a function from stage2 to stage1 to append symbols to automatically export them.
This happends under the following conditions:
- Target is wasm
- User has not provided --export/--rdynamic flags themselves.
This allows the `acid` debugger on
plan9 to be used to debug a zig source
file without patching `acid`!
The patch adds a second `z` symbol. This z
symbol has a value of 0, which means that it
pops the history stack. We put a very large
number for the value of the second symbol because
it has to be at least as large as the linecount of
the file. The debuginfo format is meant to be used
with c files, where the stack would look something
like this:
```
-> Line: 0x1 (1) Name: 0x1/0x2/0x3/0xe/0x13/0x1b (/sys/src/libc/port/malloc.c)
-> Line: 0x2 (2) Name: 0x1/0x6/0x7/0x8 (/amd64/include/u.h)
-> Line: 0x4f (79) Name: ()
-> Line: 0x50 (80) Name: 0x1/0x2/0x7/0x9 (/sys/include/libc.h)
-> Line: 0x358 (856) Name: ()
-> Line: 0x359 (857) Name: 0x1/0x2/0x7/0x1c (/sys/include/pool.h)
-> Line: 0x392 (914) Name: ()
-> Line: 0x393 (915) Name: 0x1/0x2/0x7/0x1d (/sys/include/tos.h)
-> Line: 0x3ab (939) Name: ()
-> Line: 0x4eb (1259) Name: ()
```
however in zig, we do not use includes and .h files,
so we only need the first and last items in the stack:
the source file that the symbols belong to, and the pop
symbol with a null name and a value of the total linecount of the
preprocessed source. Since there is no preprocessing in zig, we
just make the linecount very large. There do not appear to be
any downsides to this approach. If this causes a bug in the future,
a simple fix would be to make the pop symbol just have the value
of how many newlines are in the source file.
const locals now detect if the value ends up being comptime known. In
such case, it replaces the runtime AIR instructions with a decl_ref
const.
In the backends, some more sophisticated logic for marking decls as
alive was needed to prevent Decls incorrectly being garbage collected
that were indirectly referenced in such manner.
Add a variant of the `validate_struct_init` ZIR instruction:
`validate_struct_init_comptime` which is the same thing except it
indicates a comptime scope.
Sema code for this instruction now handles default struct field
values and detects when the struct initialization resulted in a
comptime value, replacing the already-emitted AIR instructions
to store each individual field with a single `store` instruction
with a comptime struct value as the operand.
In the case of a comptime scope, there is a simpler path that only
evals the implicit store instructions for default field values, avoiding
the mechanism for detecting comptime values.
This regressed one test case for the wasm backend, but it's just hitting
a different prong of `emitConstant` which currently has "TODO" in there,
so I think it's fine.
This allows Zig code to perform conditional compilation based on a tag
by which a Zig compiler implementation identifies itself.
See the doc comment in this commit for more details.
We now detect if the return type will be set by passing the first argument
as a pointer to stack memory from the callee's frame. This way, we do not have to
worry about stack memory being overwritten.
Besides this, we implement memset by either using wasm's memory.fill instruction when available,
or lower it manually. In the future we can lower this to a compiler_rt call.
Previously we were performing the wrapping and unwrapping operations incorrectly.
We now correctly create the type and set its values.
Besides this, we also set the null-byte to the incorrect value, which meant we were
doing the opposite action of a is_null check. This is now fixed as well.
While implementing this, I found a small bug in the wrapErrUnionPayload where we
would load a pointer value and save that, rather than store the pointer with the error.
This is now fixed as well, by copying the entire operand into the payload of the error union.
copy_cqes() is not guaranteed to return as many CQEs as provided in the
`wait_nr` argument, meaning the assert in `copy_cqe` can trigger.
Instead, loop until we do get at least one CQE returned.
This mimics the behaviour of liburing's _io_uring_get_cqe.
This commit fixes two problems:
* `zig build-obj` regressed from the cache-mode branch. It would crash
because it assumed that dirname on the emit bin path would not be
null. This assumption was invalid when outputting to the current
working directory - a pretty common use case for `zig build-obj`.
* When using the LLVM backend, `-fno-emit-bin` combined with any other
kind of emitting, such as `-femit-asm`, emitted nothing.
Both issues are now fixed.
Doc comments reproduced here:
This function is called by the frontend before flush(). It communicates that
`options.bin_file.emit` directory needs to be renamed from
`[zig-cache]/tmp/[random]` to `[zig-cache]/o/[digest]`.
The frontend would like to simply perform a file system rename, however,
some linker backends care about the file paths of the objects they are linking.
So this function call tells linker backends to rename the paths of object files
to observe the new directory path.
Linker backends which do not have this requirement can fall back to the simple
implementation at the bottom of this function.
This function is only called when CacheMode is `whole`.
This solves stack trace regressions on Windows and macOS because the
linker backends do not observe object file paths until flush().
Small packed structs weren't included in this resolution so their
c_abi_type would be NULL when attempting usage later, leading to a
compiler crash.
Resolves#10431.
The semantics of this function are that it moves both files and
directories. Previously we had this `is_dir` boolean field of
`std.os.windows.OpenFile` which required the API user to choose: are we
opening a file or directory? And the other kind would either cause
error.IsDir or error.NotDir. But that is not a limitation of the Windows
file system API; it was self-imposed.
On Windows, rename is implemented internally with `NtCreateFile` so we
need to allow it to open either files or directories. This is now done
by `std.os.windows.OpenFile` accepting enum{file_only,dir_only,any}
instead of a boolean.
When calling a comptime or inline function, if the parameter is generic and
is resolved to generic_poison or generic_poison_type, the invocation was
part of another function's parameters or return type expression and is
dependent on an as-of-yet type of another parameter. In this case, processing
should stop, and we return error.GenericPoison to let the caller in funcCommon,
zirParam or zirParamAnytype know that the function is generic.
This allows stage2 to build more of compiler-rt.
I also changed `-%` to `-` for comptime ints in the div and mul
implementations of compiler-rt. This is clearer code and also happens to
work around a bug in stage2.
This improves readability as well as compatibility with stage2. Most of
compiler-rt is now enabled for stage2 with just a few functions disabled
(until stage2 passes more behavior tests).
Instead of juggling GPA-allocated sub_path (and ultimately dropping the
ball, in this analogy), `Compilation.create` allocates an
already-exactly-correct size `sub_path` that has the digest unpopulated.
This is then overwritten in place as necessary and used as the
`emit_bin.sub_path` value, and no allocations/frees are performed for
this file path.
Previously the code asserted source files were already loaded, but this
is not the case when cached ZIR is loaded. Now it will trigger .zig
source code to be loaded for the purposes of hashing the source for
`CacheMode.whole`.
This additionally refactors stat_size, stat_inode, and stat_mtime fields
into using the `Cache.File.Stat` struct.
This fixes a regression in this branch that can be reproduced with the
following steps:
1. `zig build-exe hello.zig`
2. delete the "hello" binary
3. `zig build-exe hello.zig`
4. observe that the "hello" binary is missing
This happened because it was a cache hit, but nothing got copied to the
output directory.
This commit sets CacheMode to incremental - even for stage1 - when the
CLI requests `disable_lld_caching` (this option should be renamed),
resulting in the main Compilation to be repeated (uncached) for stage1,
populating the binary into the cwd as expected.
For stage2 the result is even better: the incremental compilation system
will look for build artifacts to incrementally compile, and start fresh
if not found.
when using `CacheMode.whole`. Also, I verified that `addDepFilePost` is
in fact including the original C source file in addition to the files it
depends on.
* Logic to check whether a bin file is not emitted is more complicated
in between `Compilation.create` and `Compilation.update`. Fixed the
logic that decides whether to build compiler-rt and other support
artifacts.
* Basically, one cannot inspect the value of `comp.bin_file.emit` until
after update() is called - fixed another instance of this happening
in the CLI.
* In the CLI, `runOrTest` is updated to properly use the result value
of `comp.bin_file.options.emit` rather than guessing whether the
output binary is.
* Don't assume that the emit output has no directory components in
sub_path. In other words, don't assume that the emit directory is the
final directory; there may be sub-directories.
The two CacheMode values are `whole` and `incremental`.
`incremental` is what we had before; `whole` is new.
Whole cache mode uses everything as inputs to the cache hash;
and when a hit occurs it skips everything including linking.
This is ideal for when source files change rarely and for backends that
do not have good incremental compilation support, for example
compiler-rt or libc compiled with LLVM with optimizations on.
This is the main motivation for the additional mode, so that we can have
LLVM-optimized compiler-rt/libc builds, without waiting for the LLVM
backend every single time Zig is invoked.
Incremental cache mode hashes only the input file path and a few target
options, intentionally relying on collisions to locate already-existing
build artifacts which can then be incrementally updated.
The bespoke logic for caching stage1 backend build artifacts
is removed since we now have a global caching mechanism for
when we want to cache the entire compilation, *including* linking.
Previously we had to get "creative" with libs.txt and a special
byte in the hash id to communicate flags, so that when the cached
artifacts were re-linked, we had this information from stage1
even though we didn't actually run it. Now that `CacheMode.whole`
includes linking, this extra information does not need to be
preserved for cache hits. So although this changeset introduces
complexity, it also removes complexity.
The main trickiness here comes from the inherent differences between the
two modes: `incremental` wants a directory immediately to operate on,
while `whole` doesn't know the output directory until the compilation is
complete. This commit deals with this problem mostly inside `update()`,
where, on a cache miss, it replaces `zig_cache_artifact_directory` with a
temporary directory, and then renames it into place once the compilation is
complete.
Items remaining before this branch can be merged:
* [ ] make sure these things make it into the cache manifest:
- @import files
- @embedFile files
- we already add dep files from c but make sure the main .c files make
it in there too, not just the included files
* [ ] double check that the emit paths of other things besides the binary
are working correctly.
* [ ] test `-fno-emit-bin` + `-fstage1`
* [ ] test `-femit-bin=foo` + `-fstage1`
* [ ] implib emit directory copies bin_file_emit directory in create() and needs
to be adjusted to be overridden as well.
* [ ] make sure emit-h is handled correctly in the cache hash
* [ ] Cache: detect duplicate files added to the manifest
Some preliminary performance measurements of wall clock time and
peak RSS used:
stage1 behavior (1077 tests), llvm backend, release build:
* cold global cache: 4.6s, 1.1 GiB
* warm global cache: 3.4s, 980 MiB
stage2 master branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 191 MiB
* warm global cache: 0.40s, 128 MiB
stage2 this branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 179 MiB
* warm global cache: 0.27s, 90 MiB
This saves on comptime format string parsing, as the compiler caches
comptime calls. The catch here, is that parsePlaceHolder cannot take the
placeholder string as a slice. It must take it as an array by value for
the caching to occure.
There is also some logic in here that ensures that the specifier_arg is
always them same slice when the items they contain are the same. This
makes the compiler stamp out less copies of formatType.
For renameat, unlinkat, mkdirat, symlinkat and linkat the error code
differs between kernel 5.4 which returns EBADF and kernel 5.10 which returns EINVAL.
Fixes#10466
Previously, the `load` instruction would just pass the pointer to the next instruction
for types that comply to `isByRef`. However, this meant that a defer would directly write
to the reference, rather than a copy. After this commit, we always copy the value.
- This implements all pointer arithmetic related instructions such as ptr_add, ptr_sub, ptr_elem_val
- We refactored the code, to use `isByRef` to ensure consistancy.
- Pointers will now be loaded correctly, rather then being passed around.
- The behaviour test for pointers is now passing.
- Previously the table index and function type index were switched.
This commit swaps them.
- This also emits the correct indirect function calls count when importing the function table
- Add method to easily create local for virtual stack
- Ensure function pointers are passed correctly
- Correctly handle slices as return types and values
- Fix wrapping error sets/payloads.
- Handle ptr-like optionals correctly, by using address '0' as null.
- Implement `array_to_slice`
- linker: Always emit a table, so call_indirect inside bodies do not fail if there's no table.
TODO: Only do this when we emit a call_indirect but the relocation cannot be resolved.
* load address (pointer) to a stack variable in a register via
`lea` instruction
* store value on the stack via a pointer stored in a register via
`mov [reg], imm` instruction
* the lowerings naturally are handled automatically by Mir -> Isel
layer
* add initial (without safety) implementation of `.optional_payload`
* add matching stage2 test cases
Effectively a small continuation of #10152
This allows the for.zig behavior tests to pass. Unfortunately to fully test everything I had to move a lot of behavior tests from array.zig; most of them now pass (sorry @rainbowbismuth!)
I'm also conflicted on how I store constants into arrays because it's kind of stupid; array's can't be re-initialized using the same syntax, so instead of initializing each element, a new array is made which is copied into the destination. This also required that renderValue can't emit string literals for byte arrays given that they need to always have an extra byte for the NULL terminator, meaning that strings are no longer grep-able in the output.
* fix handling of `ah`, `bh`, `ch`, and `dh` registers (which are
actually used as aliases to `dil`, etc. registers). Currenly, we
treat them as aliases only meaning when we encounter `ah` we make
sure to set the REX.W to promote the instruction to 64bits and use
`dil` register instead - otherwise we might have mismatch between
registers used in different parts of the codegen. In the future,
we can and should use `ah`, etc. as upper 8bit halves of 16bit
registers `ax`, etc.
* fix bug in `airCmp` where `.cmp` MIR instruction shouldn't force
type `Bool` but let the type of the original type propagate downwards
- we need this to make an informed choice of the target register
size and hence choose the right encoding down the line.
* implement lowering of 1-byte and 2-byte values to stack and add
matching stage2 tests for x86_64 codegen
To request memory-immediate encoding at the MIR side, we should now
use a new tag such as `mov_mem_imm` where the size of the memory
pointer is encoded as the flags:
```
0b00 => .byte_ptr,
0b01 => .word_ptr,
0b10 => .dword_ptr,
0b11 => .qword_ptr,
```
* `Module.Union.getLayout`: fixes to support components of the union
being 0 bits.
* Implement `@typeInfo` for unions.
* Add missing calls to `resolveTypeFields`.
* Fix explicitly-provided union tag types passing a `Zir.Inst.Ref`
where an `Air.Inst.Ref` was expected. We don't have any type safety
for this; these typess are aliases.
* Fix explicitly-provided `union(enum)` tag Values allocated to the
wrong arena.
* reduce number of branches in zirCmpEq
* implement equality comparison for enums and unions
* fix coercion from union to its tag type resulting in the wrong type
* fix method calls of unions
* implement peer type resolution for unions, enums, and enum literals
* fix union tag type memory in the wrong arena
Comment from this commit reproduced here:
LLVM does not allow us to change the type of globals. So we must
create a new global with the correct type, copy all its attributes,
and then update all references to point to the new global,
delete the original, and rename the new one to the old one's name.
This is necessary because LLVM does not support const bitcasting
a struct with padding bytes, which is needed to lower a const union value
to LLVM, when a field other than the most-aligned is active. Instead,
we must lower to an unnamed struct, and pointer cast at usage sites
of the global. Such an unnamed struct is the cause of the global type
mismatch, because we don't have the LLVM type until the *value* is created,
whereas the global needs to be created based on the type alone, because
lowering the value may reference the global as a pointer.
* fix initialisation of void* fields of structs (initialises to 0xaa.. rather than {})
* don't generate struct fields when the field type does not have codegen bits
* in airAlloc generate a void* literal if the element type does not have codegen bits
- neg can only overflow, if a == MIN
- case `-0` is properly handled by hardware, so overflow check by comparing
`a == MIN` is sufficient
- tests: MIN, MIN+1, MIN+4, -42, -7, -1, 0, 1, 7..
See #1290
The main problem was that the loop body was treated as an expression
that was one of the peer result values of a loop, when in reality the
loop body is noreturn and only the `break` operands are the result
values of loops.
This was solved by introducing an override that prevents rvalue() from
emitting a store to result location instruction for loop bodies.
An orthogonal change also included in this commit is switching
`elem_val` index expressions to using `coerced_ty` and doing the
coercion to `usize` inside `Sema`, resulting in smaller ZIR (since the
cast becomes implied).
I also changed the break operand expression to use `reachableExpr`,
introducing a new compile error for double break.
This makes a few more behavior tests pass for `while` and `for` loops.
* Make bcrypt State struct public
This is useful to implement the various protocols outside of the standard library
* Implement bcrypt pbkdf
This variant is used in e.g. SSH
The OpenBSD implementation was used as a reference
Introduced a new AIR instruction: `tag_name`. Reasons to do this
instead of lowering it in Sema to a switch, function call, array
lookup, or if-else tower:
* Sema is a bottleneck; do less work in Sema whenever possible.
* If any optimization passes run, and the operand to becomes
comptime-known, then it could change to have a comptime result
value instead of lowering to a function or array or something which
would then have to be garbage-collected.
* Backends may want to choose to use a function and a switch branch,
or they may want to use a different strategy.
Codegen for `@tagName` is implemented for the LLVM backend but not any
others yet.
Introduced some new `Type` tags:
* `const_slice_u8_sentinel_0`
* `manyptr_const_u8_sentinel_0`
The motivation for this was to make typeof() on the tag_name AIR
instruction non-allocating.
A bunch more enum tests are passing now.
* remove false positive "all prongs handled" compile error for
non-exhaustive enums.
* implement `@TypeInfo` for enums, except enums which have any
declarations is still TODO.
* `getBuiltin` uses nomespaceLookup/analyzeDeclVal rather than
namespaceLookupRef/analyzeLoad. Avoids a detour through an
unnecessary type, and adds a detour through a caching mechanism.
* `Value.eql`: add missing code to handle enum comparisons for
non-exhaustive enums. It works by converting the enum tags to numeric
values and comparing those.
- abs can only overflow, if a == MIN
- comparing the sign change from wrapping addition is branchless
- tests: MIN, MIN+1,..MIN+4, -42, -7, -1, 0, 1, 7..
See #1290
Layout algorithm: all `align(0)` fields are squished together as if they
were a single integer with a number of bits equal to `@bitSizeOf` each
field added together. Then the natural ABI alignment of that integer is
used for that pseudo-field.
Previously, this function would return an incorrect result for structs
and unions which did not have their fields resolved yet.
This required introducing more logic in Sema to resolve types before
doing certain things such as creating an anonmyous Decl and emitting
function call AIR.
As a result a couple more struct tests pass.
Oh, and I implemented the language change to make sizeOf for pointers
always return pointer size bytes even if the element type is 0 bits.
The linker will now emit names for all function, global and data segment symbols.
This increases the ability to debug wasm modules tremendously as tools like wasm2wat
can use this information to generate named functions, globals etc, rather than placeholders such as $f1.
This way I am hopeful they can be reused for every MIR lowering
function which follows a given encoding. Currently, support MI,
RM and MR encodings without SIB scaling.
Instead of a separate function, `coerceNum` for handling comptime-known
number coercion, outside of the main switch, the `coerce` function now
has a single big switch statement that decides the control flow based on
the zig type tag.
* Extract common logic between `zirStructInitEmpty` and
`zirStructInit`.
* `resolveTypeFields` additionally sets status to `have_layout` if the
total number of fields is 0.
Allocate a new program header and a new section to accomodate the read-only data
section ".rodata".
Separate TextBlock into multiple TextBlockList, to separate decl in different
sections.
If a Decl is not a function, it is added to the .rodata section.
While this is technically incorrect, proper handling of anyopaque, as well
as regular opaque, is probably best left until pointers to zero-sized types
having no bits is abolished.
The self-hosted compiler cannot yet deal with the print function that this
field enables. It is not critical, however, and allows us to remove formatting
from the list of neccesary features to implement to get the page allocator
working.
This caused zirParam instructions of parent blocks to be present in
inline analyzed blocks, and so function prototypes declared in the
inline blocks would also gain and add to the parameters in the
parent block.
Only block and block_inline are affected in this commit, as prototypes
and declarations are always generated in block_inline. This might need
to be resolved in a more general way at some point.
Previously, function parameter instructions for function prototypes would be
generated in the parent block. This caused issues in blocks where multiple
prototypes would be generated in, such as the block for struct fields for
example. This change introduces an inline block around every prototype such
that all parameters for a prototype are confined to a unique block.
This allows the inferred error set of comptime and inline invocations to be
resolved separately from the inferred error set of the runtime version or other
comptime/inline invocations.
* turns out MOV and other arithmetic instructions such as ADD can
naturally share the same lowering codepath (for the same variants)
* there are variants that are specific to ADD, or MOV which will be
implemented as standalone MIR tags
* tweak Isel tests to generate corresponding test cases for all
arithmetic instructions in comptime
* variant `0b11` when both `reg1 != .none` and `reg2 != .none` is
identical to `0b00` therefore it can safely be removed
* fix proper destination register size calculation when setting register from
another source register
* introduce `EmitResult` wrapper struct for easier manipulation of
intermediate emit results - this is mainly to track errors such as
size mismatch between operands
* create an informative `ErrorMsg` directly at the callsite
* ensure that every callsite of basic MOV MIR instruction follows the
Intel syntax (dst <- src)
* add extensive unit tests for MOV MIR -> Isel lowering
* leave TODOs for cases that are currently not handled and/or missing
* fix any ABI size mismatch between operands
Make `@returnAddress()` return for the BPF target, as the BPF target for
the time being does not support probing for the return address. Stack
traces for the general purpose allocator for the BPF target is also set
to not be captured.
Before, `std.Progress` was printing unwanted stuff to stderr. Now, the
test runner's logic to detect whether we should print each test as a
separate line to stderr is properly activated.
* Create Vector language documentation
Main changes to docs:
1. Create brief documentation on Zig vector types with code example
2. Get rid of the SIMD sub-heading under the main Vectors heading,
and update links accordingly
3. Add an example to the `@shuffle` docs
The status quo for the `build.zig` build system is preserved in
the sense that, if the user does not explicitly override
`dylib.setInstallName(...);` in their build script, the default
of `@rpath/libname.dylib` applies. However, should they want to
override the default behaviour, they can either:
1) unset it with
```dylib.setIntallName(null);```
2) set it to an explicit string with
```dylib.setInstallName("somename.dylib");```
When it comes to the command line however, the default is not to
use `@rpath` for the install name when creating a dylib. The user
will now be required to explicitly specify the `@rpath` as part
of the desired install name should they choose so like so:
1) with `build-lib`
```
zig build-lib -dynamic foo.zig -install_name @rpath/libfoo.dylib
```
2) with `cc`
```
zig cc -shared foo.c -o libfoo.dylib -Wl,"-install_name=@rpath/libfoo.dylib"
```
This reverts commit 11803a3a56.
Observations from the performance dashboard:
* strictly worse in terms of CPU instructions
* slightly worse wall time (but this can be noisy)
* sometimes better, sometimes worse for branch predictions
Given that the commit was introducing complexity for optimization's
sake, these performance changes do not seem worth it.
See https://github.com/ziglang/zig/pull/10337 for context.
In #10337 the `available` tracking fix necessitated an additional condition on the probe loop in both `getOrPut` and `getIndex` to prevent an infinite loop. Previously, this condition was implicit thanks to the guaranteed presence of a free slot.
The new condition hurts the `HashMap` benchmarks (https://github.com/ziglang/zig/pull/10337#issuecomment-996432758).
This commit removes that extra condition on the loop. Instead, when probing, first check whether the "home" slot is the target key — if so, return it. Otherwise, save the home slot's metadata to the stack and temporarily "free" the slot (but don't touch its value). Then continue with the original loop. Once again, the loop will be implicitly broken by the new "free" slot. The original metadata is restored before the function returns.
`getOrPut` has one additional gotcha — if the home slot is a tombstone and `getOrPut` misses, then the home slot is is written with the new key; that is, its original metadata (the tombstone) is not restored.
Other changes:
- Test hash map misses.
- Test using `getOrPutAssumeCapacity` to get keys at the end (along with `get`).
When entries are inserted and removed into a hash map at an equivalent rate (maintaining a mostly-consistent total count of entries), it should never need to be resized. But `HashMapUnmanaged.available` does not presently count tombstoned slots as "available", so this put/remove pattern eventually panics (assertion failure) when `available` reaches `0`.
The solution implemented here is to count tombstoned slots as "available". Another approach (which hashbrown (b3eaf32e60/src/raw/mod.rs (L1455-L1542)) takes) would be to rehash all entries in place when there are too many tombstones. This is more complex but avoids an `O(n)` bad case when the hash map is full of many tombstones.
This number tracks the glibc version in the oldest still-active LTS
version of Debian, which is Jessie, extended LTS expiring in June 2022,
at which point this number can be bumped again.
Before this commit, glibc headers did the following mapping:
* (zig) mipsel-linux-gnu => (glibc) mipsel-linux-gnu
* (zig) mipsel-linux-gnu-soft => (glibc) (none)
* (zig) mips-linux-gnu => (glibc) mips-linux-gnu
* (zig) mips-linux-gnu-soft => (glibc) (none)
While the glibc ABI stubs used the (zig) gnueabi and gnueabihf ABIs,
and the stage2 available_libcs array listed:
* (zig) mipsel-linux-gnu
* (zig) mips-linux-gnu
The problem is the mismatch between the ABI component of the headers and
the stubs.
This commit makes the following clarifications:
* (zig) mips-linux-gnueabi means soft-float
* (zig) mipsel-linux-gnueabi means soft-float
* (zig) mips-linux-gnueabihf means hard-float
* (zig) mipsel-linux-gnueabihf means hard-float
Consequently, the glibc headers now do this mapping:
* (zig) mips-linux-gnueabihf => (glibc) mips-linux-gnu
* (zig) mipsel-linux-gnueabihf => (glibc) mipsel-linux-gnu
* (zig) mips-linux-gnueabi => (glibc) mips-linux-gnu-soft
* (zig) mipsel-linux-gnueabi => (glibc) mipsel-linux-gnu-soft
The glibc ABI stubs are unchanged, and the stage2 available_libcs
array's 2 entries are modified and it gains 2 more:
* (zig) mipsel-linux-gnueabi
* (zig) mipsel-linux-gnueabihf
* (zig) mips-linux-gnueabi
* (zig) mips-linux-gnueabihf
Now everything is consistent. Zig no longer recognizes a `mips-linux-gnu`
triple; one must use `mips-linux-gnueabi` (soft float) or
`mips-linux-gnueabihf` (hard float).
Upstream, some of the nonshared functions moved to be different for hurd
and for linux. Since our glibc is linux-only we update to use the
linux-specific files.
This fixes std lib tests for x86_64 when linking glibc.
This commit introduces tools/update_glibc.zig to update the start files
for next time.
Some notable changes in recent glibc:
* abi-note.S has been changed to abi-note.c but we resist the change to
keep it easier to compile the start files.
* elf-init.c has been deleted upstream. Further testing should be done
to verify that binaries against glibc omitting elf-init.c still run
properly on oldel glibc linux systems.
Closes#4926
execve can return EBADLIB on Linux. I observed this when passing
an x86_64 interpreter path to qemu-i386.
This error code is Linux and Solaris-only. I came up with an improved
pattern for dealing with OS-specific error codes.
Since `BoundedArray.insert` internally reserves space for the element
to be inserted, it can support inserting at the position that is
the current length of the array. Change the check for the insertion position
to allow this.
- adds __cmpsi2, __cmpdi2, __cmpti2
- adds __ucmpsi2, __ucmpdi2, __ucmpti2
- use 2 if statements with 2 temporaries and a constant
- tests: MIN, MIN+1, MIN/2, -1, 0, 1, MAX/2, MAX-1, MAX if applicable
See #1290
- use negXi2.zig to prevent confusion with negXf2.zig
- used for size optimized builds and machines without carry instruction
- tests: special cases 0, -INT_MIN
* use divTrunc range and shift with constant offsets
See #1290
Notating a symbol to be exported in code will only tell the linker
where to find this symbol, so other object files can find it. However, this does not mean
said symbol will also be exported to the host environment. Currently, we 'fix' this by force
exporting every single symbol that is visible. This creates bigger binaries and means host environments
have access to symbols that they perhaps shouldn't have. Now, users can tell Zig which symbols
are to be exported, meaning all other symbols that are not specified will not be exported.
Another change is we now support `-rdynamic` in the wasm linker as well, meaning all symbols will
be put in the dynamic symbol table. This is the same behavior as with ELF. This means there's a 3rd strategy
users will have to build their wasm binary.
While investigating slow build times with [a large project](https://github.com/hexops/mach/issues/124),
I found that the compiler was reading from disk nearly every C source file in my project
when rebuilding despite no changes having been made. This accounted for several seconds of
time (approx. 20-30% of running `zig build` without any changes to the sources.)
The cause of this was that comparisons of file mtimes would _always_ fail (the mtime of the file on
disk was always newer than that stored in the cache manifest), and so the cache logic would always
fall back to byte-for-byte file content comparisons with what is on disk vs. in the cache-reading every
C source file in my project from disk during each rebuild. Because file contents were the same, a cache
hit occurred, and _despite the mtime being different the cache manifest would not be updated._
One can reproduce this by building a Zig project so the cache is populated, and then changing mtimes
of their C source files to be newer than what is in the cache (without altering file contents.)
The fix is rather simple: we should always write the updated cache manifest regardless of
whether or not a cache hit occurred (a cache hit doesn't indicate if a manifest is dirty) Luckily,
`writeManifest` already contains logic to determine if a manifest is dirty and becomes no-op if no
change to the manifest file is necessary-so we merely need to ensure it is invoked.
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
When the Zig compiler is statically linked, it inspects the
/usr/bin/env ELF file to determine the native glibc version, by checking
the DT_RUNPATH, and then calling readlink() on the libc.so file, because
typically the symlink will have e.g. libc-2.33.so in the name, revealing
the glibc version.
Fortunately, this information is also in readlink() of ld.so, which is
available as the "INTERP" file path. This commit looks for e.g.
`ld-2.33.so` on the symlink data for the dynamic linker.
In theory a more complete solution would also look at `/etc/ld.so.cache`
if necessary, and finally fall back to some hard coded paths, in order
to resolve the location of libc.so, in order to do this readlink() trick
on the resulting path. You can find that flow chart with `man ld.so`.
But I think this logic will be enough to get a correct answer in all real
world cases.
This has been tested on Debian Buster and glibc-based Void Linux.
Fixes#6469
This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/
The abilists file is different in these ways:
* It additionally encodes whether a symbol is a function or an object,
and if it is an object, it additionally encodes the size in bytes.
* It additionally encodes migrations of symbols from one library to
another between glibc versions.
* It is binary data instead of ascii.
* It is one file instead of three.
* It is 165 KB instead of 200 KB.
This solves three bugs:
Fixes#7667Fixes#8714Fixes#8896
This way, we will inform the user that there are unresolved symbols
in addition to missing library/framework as requested on the linker
line. If all symbols were resolved on the other hand, we still
flag up that the library/framework cannot be found.
Example behaviour:
```
$ zig cc hello.c -framework MyFoundation --verbose
warning(link): framework not found for '-framework MyFoundation'
warning(link): Framework search paths:
warning(link): /Library/Frameworks
warning(link): /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
thread 1079397 panic: attempt to unwrap error: FrameworkNotFound
...stack trace...
```
and
```
❯ zig cc hello.c -lWAT --verbose
warning(link): library not found for '-lWAT'
warning(link): Library search paths:
warning(link): /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
warning(link): /usr/local/lib
thread 1079824 panic: attempt to unwrap error: LibraryNotFound
...stack trace...
```
We now respect both `-fallow-shlib-undefined` and
`-Wl,"-undefined=dynamic_lookup"` flags. This is the first step
towards solving issues #8180 and #3000. We currently do not expose
any other ld64 equivalent flag for `-undefined` flag - we basically
throw an error should the user specify a different flag. Support for
those is conditional on closing #8180. As a result of this change,
it is now possible to generate a valid native Node.js addon with Zig
for macOS.
- each byte gets masked, shifted and combined
- use boring masks instead of comptime for readability
- tests: bit patterns with reverse operation, if applicable
See #1290
The previous commit tried to use atomics but not many CPUs support
128-bit atomics. So we use a mutex. In order to avoid contention, we
also store `recent_problematic_timestamp` locally on the `Manifest`
which is only ever accessed from a single thread at a time, and only
consult the global one if the local one is problematic.
This commit was tested by running `zig build test-behavior` in two
separate terminals at the same time.
Previously `recent_problematic_timestamp` was unprotected and accessed
potentially with multiple worker threads simultaneously.
This commit protects it with atomics and also introduces a flag to
prevent multiple timestamp checks from within the same call to hit().
Unfortunately the compiler-rt function __sync_val_compare_and_swap_16 is
not yet implemented, so I will have to take a different strategy in a
follow-up commit.
* put `recent_problematic_timestamp` onto `Cache` so that it can be
shared by multiple Manifest instances.
* make `isProblematicTimestamp` return true on any filesystem error.
* save 1 syscall by using truncate=true in createFile instead of
calling `setEndPos`.
1. It was looking for trailing zero bits when it should be looking for
trailing decimal zeros.
2. Clock timestamps had more precision than the actual file timestamps
The fix is to grab a timestamp from a 'just now changed' temp file.
This timestamp is "problematic". Any file timestamp greater than or equal
to this timestamp is considered problematic. File timestamps **prior** to
this **can** be trusted.
Downside is that it causes a disk I/O to write to and then read the
timestamp from this file ~1ms on my system. This is partially mitigated by
keeping track of the most recent problematic timestamp, and only checking
for a new problematic timestamp when checking a timestamp that is equal to
or larger than the last problematic one.
This fixes#6082.
This is the result of the work on tools/gen_stubs.zig. It now uses the
preprocessor to emit different symbols and sizes depending on the
architecture. The data is collected directly from multiple libc.so files
on disk built with upstream musl.
Closes#8178
Addresses #8896 for musl
There is still room for further improvement to this, which is to
put `.ds` directives after symbols that are not followed by aliases, to
avoid the potential problem of a linker believing that all symbols are
aliases of each other.
Now it outputs libc.S which can be assembled with zig, and the small
differences per-architecture are handled with preprocessor directives.
There is also now a set of blacklisted symbols which contains
compiler-rt.
tools/gen_stubs.zig now cuts out the middle man and operates directly on
the libc.so ELF file. it outputs accurate .size directives for objects.
std.elf gains an STV enum.
The previous commit (38b2d62092) regressed
the compile error test case for when doing saturating shift left of a
comptime-known negative RHS.
This commit additionally fixes the error for regular shifts in addition
to saturating shifts.
Saturating shift left (`<<|`) previously used the `ir_analyze_bin_op_math`
codepath rather than the `ir_analyze_bit_shift` codepath, leading to it
doing peer type resolution (incorrect) instead of using the LHS type as
the number of bits to do the saturating against.
This required implementing SIMD vector support for `@truncate`.
Additionall, this commit adds a compile error for saturating shift left
on a comptime_int.
stage2 does not pass these new behavior tests yet.
closes#10298
All Zig code is eligible to `@import("builtin")` which is mapped to a
generated file, build.zig, based on the target and other settings.
Zig invocations which share the same target settings will generate the
same builtin.zig file and thus the path to builtin.zig is in a shared
cache folder, and different projects can sometimes use the same file.
Before this commit, this led to race conditions where multiple
invocations of `zig` would race to write this file. If one process
wanted to *read* the file while the other process *wrote* the file, the
reading process could observe a truncated or partially written
builtin.zig file.
This commit makes the following improvements:
- limitations:
- avoid clobbering the inode, mtime in the hot path
- avoid creating a partially written file
- builtin.zig needs to be on disk for debug info / stack trace purposes
- don't mark the task as complete until the file is finished being populated
(possibly by an external process)
- strategy:
- create the `@import("builtin")` `Module.File` during the AstGen
work, based on generating the contents in memory rather than
loading from disk.
- write builtin.zig in a separate task that doesn't have
to complete until the end of the AstGen work queue so that it
can be done in parallel with everything else.
- when writing the file, first stat the file path. If it exists, we are done.
- otherwise, write the file to a temp file in the same directory and atomically
rename it into place (clobbering the inode, mtime in the cold path).
- summary:
- all limitations respected
- hot path: one stat() syscall that happens in a worker thread
This required adding a missing function to the standard library:
`std.fs.Dir.statFile`. In this commit, it does open() and then fstat()
which is two syscalls. It should be improved in a future commit to only
make one.
Fixes#9439.
Not sure why this warning is being emitted; let's reexamine it on the
next libunwind upgrade. I triggered it with this:
zig c++ -o hello hello.cpp -target x86_64-windows
The console test# label [test#/#tests] was being generated inside
refreshWithHeldLock (in lib/std/Progress.zig), using the number of
completed items. This was being incremented by 1 when displayed,
which is not required.
- Correctly load slice value on stack
- Implement WrapErrorUnionErr and payload
- Implement trunc, fix sliceLen and write undefined
- Implement slice as return type and argument
Note: This also fixes a memory leak for inferred error sets, and for usingnamespace
This reverts commit 725267f7c2, reversing
changes made to 2dae860de3.
This test is failing:
```zig
pub fn main() u8 {
var e = foo();
const i = e catch 69;
return i;
}
fn foo() anyerror!u8 {
return 5;
}
```
It's returning 69 instead of the expected value 5.
Adds the `tcflag_t` type to the termios constants.
This is made to allow bitwise operations on the termios
constants without an integer cast, e.g.:
```zig
var raw = try std.os.tcgetattr(std.os.STDIN_FILENO);
raw.lflag &= std.os.linux.ECHO | std.os.linux.ICANON;
```
instead of
```zig
var raw = try std.os.tcgetattr(std.os.STDIN_FILENO);
raw.lflag &= ~@intCast(u32, std.os.linux.ECHO | std.os.linux.ICANON);
```
Contributes to #10181
This branch introduced std.Target.TargetAbi when we already had
std.Target.Abi which was, unsurprisingly, already suited for this task.
Also pull out the -mabi= cc flag addition to the common area instead of
duplicating it for assembly and c files.
The target abi can also be set in build.zig via LibExeObjStep.target_abi
The value passed in is checked that it is a valid value in
std.Target.TargetAbi
The target abi is also validated against the target cpu
I'm working on a build.zig file where I'm leveraging InstallRawStep but I'd like to change the install dir. This allows the install dir to be changd and also enhances InstallRawStep to add more options in the future by putting them into a struct with default values. This also removes the need for an extra addInstallStepWithFormat function in build.zig.
`getExternalExecutor` is moved from `std.zig.CrossTarget` to
`std.zig.system.NativeTargetInfo.getExternalExecutor`.
The function also now communicates a bit more information about *why*
the host is unable to execute a binary. The CLI is updated to report
this information in a useful manner.
`getExternalExecutor` is also improved to detect such patterns as:
* x86_64 is able to execute x86 binaries
* aarch64 is able to execute arm binaries
* etc.
Added qemu-hexagon support to `getExternalExecutor`.
`std.Target.canExecBinaries` of is removed; callers should use the more
powerful `getExternalExecutor` instead.
Now that `zig test` tries to run the resulting binary no matter what,
this commit has a follow-up change to the build system and docgen to
utilize the `getExternalExecutor` function and pass `--test-no-exec`
in some cases to avoid getting the error.
Additionally:
* refactor: extract NativePaths and NativeTargetInfo into their own
files named after the structs.
* small improvement to langref to reduce the complexity of the `callconv`
expression in a couple examples.
Previously when using `zig run` or `zig test`, zig would try to guess
whether the host system was capable of running the target binaries. Now,
it will always try. If it fails, then Zig emits a helpful warning to
explain the probable cause.
from zig-specific options to generally recognized zig build options that
any project can take advantage of. See the updated usage text for more
details.
dd62a6d2e8 short-circuited the logic of
`asmExpr` by emitting ZIR for `@compileError("...")`. This caused false
positive "unreachable code" errors for stage1 when there was an
expression in the asm template.
This commit makes such cases instead go through logic of `asmExpr` like
normal, however the asm template is set to 0. This is then picked up in
Sema (part of stage2, not stage1) and reported as "assembly code must
use string literal syntax".
The old test "timeout_link_chain1" was ported from liburing test_timeout_link_chain1
509873c445/test/link-timeout.c (L539-L628)
However it turns out that both fails with EBADF (-9) on Linux kernel 5.4.
The this new test skips properly on Linux kernel 5.4
and passes on Linux kernel 5.11.
- use Bit Twiddling Hacks: Compute parity in parallel
- test cases derived from popcount.zig
- tests: compare naive approach 10_000 times with random numbers created
from naive seed 42
- compiler_rt.zig: sort by LLVM builtin order and add comments to improve structure
See #1290
The end-game for inline assembly is that the syntax is more integrated
with zig, and it will not allow string concatenation for the assembler
code, for the same reasons that Zig does not have a preprocessor.
However, inline assembly in zig right now is lacking for a variety of
use cases (take a look at the open issues having to do with inline
assembly for example), and being able to use comptime expressions to
concatenate text is a workaround that real-world users are exploiting to
get by in the short term.
This commit keeps "assembly code must use string literal syntax" as a
compile error when using stage2, but allows it through when using
stage1.
I expect to revert this commit after making enough improvements to
inline assembly that our real world users' needs are satisfied.
Previously there was only `--single-threaded`.
This flag now matches other boolean flags, instead of only being able to
opt in to single-threaded builds, you can now force multi-threaded
builds. Currently this only has the possibility to emit an error
message, but it is a better user experience to understand why one cannot
choose to enable threads in some cases.
This is breaking change to the CLI.
Related: #10143
The INVAL error was marked unreachable which prevents handling
of the error at a higher level.
It seems like it should map to BadPathError based on the man page for
rmdir (and an incomplete understanding of DeleteDirError), which says:
```
EINVAL pathname has . as last component.
```
This mostly reverts commit 692c254336.
The test "for loop over pointers to struct, getting field from struct
pointer" is still failing on the CI so that one is not moved over.
similar commit from the past:
c73cd05468
This also modifies tools/update-linux-headers.zig to remove these same
files for next time to prevent a regression.
closes#10249
I don't think we can guarantee that especially for system dyld
dylibs which can be loaded at any address (perhaps even some
OS preferential low memory address).
Incidentally, this fixes stack trace tests on x86_64 macOS 12.
In order to be linker-independent, when parsing debug info in each
linked OSO, we also create a quick lookup table for symbols defined
within the OSO. We then use this lookup to map symbol from the EXE
to its defined address within the original OSO which we can then
use to extract its associated DWARF info (if any).
If a symbol is undefined after we tried resolving it in static and
dynamic libraries, and it is annotated with N_DESC_DISCARDED flag,
we simply ignore it rather than flagging an undefined symbol error.
This reverts commit 0a9b4d092f.
Hm, these are all passing for me locally. I'll have to do some
troubleshooting to figure out which one(s) are failing on the CI.
* Add missing Linux headers. Closes#9837
* Update existing headers to latest Linux.
* Consolidate headers that are the same for multiple Zig target CPU
architectures. For example, Linux has only an x86 directory for both
x86_64 and x86 CPU architectures. Now Zig only ships an x86 directory
for Linux headers, and will emit the proper corresponding -isystem
flags.
* tools/update-linux-headers.zig is now available for upgrading to
newer Linux headers, and the update process is now documented on the
wiki.
- apply simpler approach than LLVM for __popcountdi2
taken from The Art of Computer Programming and generalized
- rename popcountdi2.zig to popcount.zig
- test cases derived from popcountdi2_test.zig
- tests: compare naive approach 10_000 times with
random numbers created from naive seed 42
See #1290
Add additional search paths pointing at homebrew prefixes as Apple
doesn't ship a static libncurses for linking - only a stub for dynamic
linking `libncurses.tbd`.
If `r_extern == 0` (the relocation is non-extern, meaning it targets
a specific memory offset within the object's section) and if the
relocation type signifies that the relocation requires correction
for RIP such as SIGNED_1, then we need to subtract the correction,
here 1 for SIGNED_1, from the calculated addend value as it's
implicitly included.
- This implements the required codegen for decl types such as pointers, arrays, structs and more.
- Wasm's start function can now use both a 'u8' and 'void' as return type. This will help us with writing tests
using the stage2 testing backend. (Until all tests of behavioural tests pass).
- Now correctly generates relocations for function pointers.
- Also implements unwrapping error union error, as well as return pointers.
The function table contains all function pointers that are called
by using call_indirect. During codegen, we create a relocation
where the linker will resolve the correct index into the table and stores
this value within the data section at the location of the pointer.
This contains a few additions:
- Proper stack pointer calculation keeping alignment in mind.
- Setting up memory layout (including user flags).
- Export or import memory
- Handle 'easy' linker tasks during incremental compilation, while offloading
heavy-tracking/computation tasks to `flush()`
- This architecture allows us to easily integrate with the rest of 'zwld' to
implement linking stage2 code with external object files.
We now resolve relocations for globals, memory addresses and function indexes.
Besides above, we now also emit imported functions correctly and create a
corresponding undefined symbol for it, where as we create a defined symbol
for all other cases.
TODO: Make incrememental compilation work again with new linker infrastructure
- Converts previous `DeclBlock` into `Atom`'s to also make them compatible when
the rest of zlwd gets upstreamed and we can link with other object files.
- Resolves function signatures and removes any duplicates, saving us a lot of
potential bytes for larger projects.
- We now create symbols for each decl of the respective type
- We can now (but not implemented yet) perform proper relocations.
- Having symbols and segment_info allows us to create an object file
for wasm.
* stage1: change the `@typeName` of `@TypeOf(undefined)`,
`@TypeOf(null)`, and `@TypeOf(.foo)` to match stage2.
* move passing behavior tests to the passing-for-stage2 section.
Previously, when a coercion needed to be inserted into a break
instruction, the `br` AIR instruction would be rewritten so that the
block operand was a sub-block that did the coercion. The problem is that
the sub-block itself was never added to the parent block, resulting in
the `br` instruction operand being a bad reference.
Now, the `br` AIR instruction that needs to have coercion instructions
added is replaced with the sub-block itself with type `noreturn`, and
then the sub-block has the coercion instructions and a new `br`
instruction that breaks from the original block.
LLVM backend needed to be fixed to lower `noreturn` blocks without
emitting an unused LLVM basic block.
Closes#7356
I did this as a patch to the source rather than passing flags so that
it would intentionally be reverted when we update to the next release of
mingw-w64. At this time if any warnings are still emitted we should find
out why and make sure upstream is aware of the problem.
If a partial read occurs past the halfway point, buf.len - i will be
less than i, which is illegal. The end bound is also entirely unecessary
in this case, so just remove it.
Since we are already detecting the path to the native SDK,
if available, also fetch SDK's version and route that to the linker.
The linker can then use it to correctly populate LC_BUILD_VERSION
load command.
Due to a deficiency in LLD, we need to special-case BPF to a simple
file copy when generating relocatables. Normally, we would expect
`lld -r` to work. However, because LLD wants to resolve BPF relocations
which it shouldn't, it fails before even generating the relocatable.
Co-authored-by: Matthew Knight <mattnite@protonmail.com>
After extern enums were removed, stage1 was left in an incorrect state
of checking for `extern enum` for exported enums. This commit fixes it
to look for an explicit integer tag type instead, and adds test coverage
for the compile error case as well as the success case.
closes#9498
This is mainly because arm64 macOS doesn't support all
versions supported by x86_64 macOS. This is just a temporary
thing until both architectures support the same set of OSes.
Previously when using `--enable-cache` and creating a Windows DLL,
without overriding the `-femit-implib` option, Zig would incorrectly
dump the .lib file to the current working directory, rather than
outputting it into the artifact directory, next to the .dll file.
Fixed.
* Fixes#8810.
* Prevent a single-line container declaration if it contains a comment
or multiline string.
* If a container declaration cannot be single-line, ensure container
fields are rendered with a trailing comma.
* If `Space.comma` is passed to `renderExpressionComma` or
`renderTokenComma`, and there already exists a comma in the source,
then render one comma instead of two.
When adding test coverage, I noticed an inconsistency in which source
location the compile error was pointing to for `@embedFile` errors vs
`@import` errors. They now both point to the same place, the string
operand.
closes#9404closes#9939
In 7e23b3245a I made -O flags to the
linker emit a warning that the argument does nothing. That was not
correct however; LLD does have some logic that does different things
depending on -O0, -O1, and -O2. It defaults to -O1, and it does less
optimizations with -O0 and more with -O2.
With this commit, e.g. `-Wl,-O1` is supported by the `zig cc` frontend,
and by default we pass `-O0` to LLD in debug mode, and `-O3` in release
modes.
I also fixed a bug in the LLD ELF linker line which was incorrectly
passing `-O` flags instead of `--lto-O` flags for LTO.
* Improve the logic for determining whether emitting an import lib is
eligible, and improve the error message when the user provides
contradictory arguments.
* Integrate with the EmitLoc / Emit system that already exists, and use
the `-femit-implib[=path]`/`-fno-emit-implib` convention that already
exists.
* Proper integration with the caching system.
* CLI: fix bug in error reporting for resolving EmitLoc values for
other parameters.
This mechanism for sending arbitrary linker args to LLD has no place in
the Zig frontend, because our goal is for the frontend to understand all
the arguments and not treat linker args like a black box.
For example we have self-hosted linking in addition to LLD, so we want to
have the options make sense to both linking codepaths, not just the LLD one.
Passing -O linker args will now result in a warning that the arg does
nothing.
We assume we are compiled on a base-2 radix floating point system. This
is a reasonable assumption. musl libc as an example also assumes this.
We implement scalbn as an alias for ldexp, since ldexp is defined as 2
regardless of the float radix. This is opposite to musl which defines
scalbn in terms of ldexp.
Closes#9799.
Handle clang's linker flag `-weak_framework` as a standard framework to
link. This requires further investigation especially to do with weak
imports and how to tie one with the other.
* add support for compiling Objective-C++ code
Prior to this change, calling `step.addCSourceFiles` with Obj-C++ file extensions
(`.mm`) would result in an error due to Zig not being aware of that extension.
Clang supports an `-ObjC++` compilation mode flag, but it was only possible to use
if you violated standards and renamed your `.mm` Obj-C++ files to `.m` (Obj-C) to
workaround Zig being unaware of the extension.
This change makes Zig aware of `.mm` files so they can be compiled, enabling compilation
of projects such as [Google's Dawn WebGPU](https://dawn.googlesource.com/dawn/) using
a `build.zig` file only.
Helps hexops/mach#21
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
* test/standalone: add ObjC++ compilation/linking test
Based on the existing objc example, just tweaked for ObjC++.
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
* Introduce a mechanism into Sema for emitting a compile error when an
integer is too big and we need it to fit into a usize.
* Add `@intCast` where necessary
* link/MachO: fix an unnecessary allocation when all that was happening
was appending zeroes to an ArrayList.
* Add `error.Overflow` as a possible error to some codepaths, allowing
usage of `math.intCast`.
closes#9710
Ensure all previous test cases are still passing, as well as add some basic tests for now
for testing pointers to the stack.
This means we can start implementing wasm's C ABI found at: https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md
We also simplified the block logic by always using 'void' block types and instead writing the value to a local,
which can then be referenced by continues instructions, as done currently by AIR.
Besides this, we also no longer need to insert blocks at an offset, as we simply write the saved temporary
after we create the block.
Rather than writing the alignment in its natural form, wasm binaries encode the alignment of types as the exponent of a power of 2.
So rather than performing this encoding during AIR->MIR, we do this while emitting MIR->binary encoding.
This allows us to keep alignment logic to its natural form while doing calculations (Which is what we need during linking as well).
We also implement optionals and pointers to an optional.
This implements basic calling convention resolving. This means that for
types such as an error union, we will now allocate space on the stack to store the result.
This result will then be saved in a temporary local at the callsite.
By calculating the abi size of the struct, we move the stack
pointer and store each field depending on its size (i.e. a 1-byte field will use i32.store8).
This commit adds all required opcodes to perform those stores and loads.
This also gets rid of `mir_offset` as we now save results of binary operations into
locals and emit its result onto the stack within condbr instead. This makes everything a lot
simpler but also more robust.
In the future, we could look into an algorithm to re-use such locals.
For struct fields we use the new `local_with_offset` tag. This stores the struct's
stack pointer as well as the field's offset from that stack pointer.
`allocLocal` will now always allocate a single local, using a given type.
All non-temporary locals will now use stack memory.
When `airAlloc` is called, we create a new local, move the stack pointer,
and write its offset into the local. Arguments act as a register and do not
use any stack space.
We no longer use offsets for binary operations, but instead write the result
into a local. In this case, the local is simply used as a register, and does not
require stack space. This allows us to ensure the order of instructions is correct,
and we no longer require any patching/inserting at a specific offset.
print_air was missing the logic to print the type of a `ty_str`.
The self-hosted wasm linker now emits a mutable global.
This entry represents the stack pointer, which has an initial value of offset table size + data size + stack size.
Stack size can either be set by the user, or has the default of a single wasm page (64KiB).
After this change, the default for dynamic libraries (`-l` or
`--library`) is to only link them if they end up being actually used.
With the Zig CLI, the new options `-needed-l` or `--needed-library` can
be used to force link against a dynamic library.
With `zig cc`, this behavior can be overridden with `-Wl,--no-as-needed`
(and restored with `-Wl,--as-needed`).
Closes#10164
Because `.buffer` is an inline array field, we actually require `self` to be passed as a pointer.
If the compiler decided to pass the object by copying, we would return a pointer to to-be-destroyed stack memory.
This whole thing needs to be reworked but for now at least don't cause a
compile error when building for a target that doesn't have stderr or
detectTTYConfig.
Previously, we have confused callee-saved with caller-saved registers
(the actual register sets were swapped). This commit fixes that
for both `.x86` and `.x86_64` native backends.
This commit also fixes the register allocation logic in `genBinMathOp`
for `.x86_64` native backend where in a situation such that we require
to spill a register, we would end up spilling the register that is
already involved in the instruction as the other operand. In such a
case, we make a note of this and spill a subsequent register instead.
The BPF target does not support mutable global variables. Mark the BPF
target as a target that does not support atomic variables in order to
avoid including the global spinlock table provided in compiler_rt.
-airLoad and airStore now properly report an error if they are used with an array, instead of having the C compiler emit a vague error
-airStoreUndefined now works with array types
-structFieldPtr now works with array types, allowing generics' tests to pass
-add additional test cases that were found to be passing
-add basic int128 test cases which previously did not pass but weren't covered
-most test cases in cast.zig now pass
-i128/u128 or smaller int constants can now be rendered
-unsigned int constants are now always suffixed with 'u' to prevent random compile errors
-pointers with a val tag of 'zero' now just emit a 0 constant which coerces to the pointer type and fixes some warnings with ordered comparisons
-pointers with a val tag of 'one' are now casted back to the pointer type
-support pointers with a u64 val
-fix bug where rendering an array's type will emit more indirection than is needed
-render uint128_t/int128_t manually when needed
-implement ptr_add/sub AIR handlers manually so they manually cast to int types which avoids UB if the result or ptr operand is NULL
-implement airPtrElemVal/Ptr
-airAlloc for arrays will not allocate a ref as the local for the array is already a reference/pointer to the array itself
-fix airPtrToInt by casting to the int type
A new print field is added to RunStep that will control whether it prints the cmd before running it. By default it will be set to builder.verbose which means it will print only if builder.verbose is true.
The stack has been adjusted so that instead of pushing to index 0 in the
integer we push to the current end/index of the underlying integer. This
means we don't require a shift for every limb after each push/pop and
instead only require a mask/or and add/sub on a single element of the array.
Fixes#5959.
This test can fail due to a fix in musl for 32 bit MIPS, where musl changes limits greater than -1UL/2 to RLIM_INFINITY.
See http://git.musl-libc.org/cgit/musl/commit/src/misc/getrlimit.c?id=8258014fd1e34e942a549c88c7e022a00445c352
Depending on the system where the test is run getrlimit can return
RLIM_INFINITY for example if RLIMIT_MEMLOCK is bigger than ~2GiB.
If that happens, the setrlimit call will fail with PermissionDenied.
The symbol "_tls_index" gets lost when using LTO.
Disabling LTO on the object file that defines it allows the link to work.
fixes https://github.com/ziglang/zig/issues/8531
* wasm: Move wasm's codegen to arch/wasm/CodeGen.zig
* wasm: Define Wasm's Mir
This declares the initial most-used instructions for wasm as
well as the data that represents them.
TODO: Add binary operand opcodes.
By re-using the wasm opcode values, we can emit each opcode very easily
by simply using `@enumToInt()`. However, this poses a possible problem:
If we use all of wasm's opcodes, it leaves us no room to use synthetic opcodes such as debugging instructions.
We could use reserved opcodes, but the wasm spec may use them at some point.
TODO: Check if we should perhaps use a 16bit tag where the highest bits are used for synthetic opcodes.
* wasm: Define basic Emit structure
* wasm: Implement corresponding Emit functions for MIR
* wasm: Initial lowering to MIR
- This implements lowering to MIR from AIR for storing and loading of locals
as well as emitting immediates.
- Relocating function indexes has been simplified a lot as well as we no
longer need to patch offsets and we write a relocatable value instead.
- Locals are now emitted at the beginning of the function section entry
meaning all offsets we generate are stable.
* wasm: Lower all AIR instructions to MIR
* wasm: Implement remaining MIR instructions
* wasm: Fix function relocations
* wasm: Get all tests working
* wasm: Make `Data` 4 bytes instead of 8.
- 64bit immediates are now stored in 2 seperate u32's.
- 64bit floats are now stored in 2 seperate u32's.
- `mem_arg` is now stored as a seperate payload in extra.
* add team_info, area_info
* update signature for get_next_image_info
* add error checks for haiku system calls
* update and cleanup of haiku constants
This commit makes airStore() handle undefined values directly instead of
delegating to renderValue(): the call to renderValue() happens too late,
when "dest = " has already been written to the stream, at which point
there's no sane way to initialize e.g. struct values by assignment.
Instead, we make airStore() use memset(dest, 0xaa, sizeof(dest)), which
should transparently handle all types.
Also moves the newly-passing tests to the top of test/behavior.zig.
1. Changed Zig pointers to functions to be typedef'd so then we can
treat them the same as other types.
2. Distinguished between const slices (zig_L prefix) and mut slices
(zig_M prefix).
3. Changed lowering of Zig "const pointers" (e.g. *const u8) to to C
"pointers to const" (e.g. const char *) rather than C "const
pointers" (e.g. char * const)
4. Ensured that all typedefs are "linked" even if the decl doesn't
require any forward declarations
5. Added test that exercises function pointer type rendering
6. Changed .slice_ptr instruction to allocate pointer local rather than
a uintptr_t local
This effectively allows us to compile
```zig
pub fn main() void {}
```
which then calls into `std.start`.
Changes required to make this happen:
* handle signed int to immediate in x86_64 and aarch64 codegen
* ensure that on arm64 macOS, `.x19` is a caller-preserved register -
I'm not sure about that one at all and would like to brainstorm it
with anyone interested and especially Joachim.
* finally, fix a bug in the linker - mark new got entry as dirty upon
atom growth.
New AIR instruction: `optional_payload_ptr_set`
It's like `optional_payload_ptr` except it sets the non-null bit.
When storing to the payload via a result location that is an optional,
`optional_payload_ptr_set` is now emitted. There is a new algorithm in
`zirCoerceResultPtr` which stores a dummy value through the result
pointer into a temporary block, and then pops off the AIR instructions
from the temporary block in order to determine how to transform the
result location pointer in case any in-between coercions need to happen.
Fixes a couple of behavior tests regarding optionals.
This is a breaking change. Before, usage looked like this:
```zig
const held = mutex.acquire();
defer held.release();
```
Now it looks like this:
```zig
mutex.lock();
defer mutex.unlock();
```
The `Held` type was an idea to make mutexes slightly safer by making it
more difficult to forget to release an aquired lock. However, this
ultimately caused more problems than it solved, when any data structures
needed to store a held mutex. Simplify everything by reducing the API
down to the primitives: lock() and unlock().
Closes#8051Closes#8246Closes#10105
--import-memory import memory from the environment
--initial-memory=[bytes] initial size of the linear memory
--max-memory=[bytes] maximum size of the linear memory
--global-base=[addr] where to start to place global data
See #8633
Systems with multiple LLVM toolchains installed (e.g. one globally and one
in $HOME/local) would get confused and fail to compile. Being explicit
about the version required will force CMake to find the right version of LLVM.
On some systems, the type of the length of a slice is different from the
nfds_t type, so cast the slice length to nfds_t. This is already done in
poll, so just copy that implementation for ppoll.
1. Function signatures that return a no member struct return void
2. Undefined var decls don't get a value generated for them
3. Don't generate bitcast code if the result isn't used, since
bitcast is a pure function. Right now struct handling code
generates some weird unused bitcast AIR, and this optimization
side steps that issue.
* incorporate Andrew's MIR draft as Mir.zig
* add skeleton for Emit.zig module - Emit will lower MIR into
machine code or textual ASM.
* implement push
* implement ret
* implement mov r/m, r
* implement sub r/m imm and sub r/m, r
* put encoding common ops together - some ops share impl such as
MOV and cmp so put them together and vary the actual opcode
with modRM ext only.
* implement pop
* implement movabs - movabs being a special-case of mov not
handled by general mov MIR instruction due to requirement to
handle 64bit immediates.
* store imm64 as a struct `Imm64{ msb: u32, lsb: u32 }` in extra data
for use with for instance movabs inst
* implement more mov variations
* implement adc
* implement add
* implement sub
* implement xor
* implement and
* implement or
* implement sbb
* implement cmp
* implement lea - lea doesn't follow the scheme as other inst above. Similarly, I
think bit shifts and rotates should be put in a separate basket too.
* implement adc_scale_src
* implement add_scale_src
* implement sub_scale_src
* implement xor_scale_src
* implement and_scale_src
* implement or_scale_src
* implement sbb_scale_src
* implement cmp_scale_src
* implement adc_scale_dst
* implement add_scale_dst
* implement sub_scale_dst
* implement xor_scale_dst
* implement and_scale_dst
* implement or_scale_dst
* implement sbb_scale_dst
* implement cmp_scale_dst
* implement mov_scale_src
* implement mov_scale_dst
* implement adc_scale_imm
* implement add_scale_imm
* implement sub_scale_imm
* implement xor_scale_imm
* implement and_scale_imm
* implement or_scale_imm
* implement sbb_scale_imm
* implement cmp_scale_imm
* port bin math to MIR
* backpatch stack size into prev MIR inst
* implement Function.gen() (minus dbg info)
* implement jmp/call [imm] - we can now call functions using indirect absolute
addressing, or via registers.
* port airRet to use MIR
* port airLoop to use MIR
* patch up performReloc to use inst indices
* implement conditional jumps (without relocs)
* implement set byte on condition
* implement basic lea r64, [rip + imm]
* implement calling externs
* implement callq in PIE
* implement lea RIP in PIE context
* remove all refs to Encoder from CodeGen
* implement basic imul ops
* pass all Linux tests!
* enable most of dbg info gen
* generate arg dbg info in Emit
No security implications, but the current hash-to-curve standard
defines the sign of the Y coordinate to be negative if `gx1`
is a square, positive otherwise.
We were doing it the other way round.
GetCurrentDirectory returns a path with a trailing slash iff the cwd is
a root directory, making the code in `resolveWindows` return an invalid
path with two consecutive slashes.
Closes#10093
Because ArrayList.initCapacity uses 'precise' capacity allocation, this should save memory on average, and definitely will save memory in cases where ArrayList is used where a regular allocated slice could have also be used.
LLVM and compiler-rt must agree on how the parameters are passed, it
turns out that in LLVM13 something changed and broke the test case for
AArch64 systems.
It has nothing to do with fma at all.
Closes#9900
* CBE: mark call.zig tests as passing
* CBE: mark enum.zig tests as passing
* CBE: mark defer.zig tests as passing
* CBE: mark hasdecl.zig tests as passing
* CBE: mark hasfield.zig tests as passing
* CBE: mark ptrcast.zig tests as passing
* CBE: mark bitcast.zig tests as passing
* CBE: mark pub_enum.zig tests as passing
* CBE: mark underscore.zig tests as passing
* CBE: mark usingnamespace.zig tests as passing
* CBE: mark bugs/655.zig tests as passing
* CBE: mark bugs/679.zig tests as passing
* CBE: mark bugs/704.zig tests as passing
* CBE: mark bugs/1486.zig tests as passing
* CBE: mark bugs/2346.zig tests as passing
* CBE: mark bugs/2889.zig tests as passing
* CBE: mark bugs/4560.zig tests as passing
* CBE: mark bugs/4769_a.zig tests as passing
* CBE: mark bugs/4769_b.zig tests as passing
* CBE: mark bugs/6850.zig tests as passing
A new zsf-hosted server is being brought online.
This should be enough for linux build, test and packaging pipeline.
Currently this pipeline does not not post artifacts.
If these functions are called more than once, then the array list would no longer be guaranteed to have enough capacity during the appendAssumeCapacity calls. With ensureUnusedCapacity, they will always be guaranteed to have enough capacity regardless of how many times the function is called.
These calls are all late-initialization of ArrayList's that were initialized outside the current scope. This allows us to still get the potential memory-saving benefits of the 'precision' of initCapacity.
initCapacity did and still does use the ensureTotalCapacityPrecise logic because the initial capacity of an ArrayList is not important in terms of how it grows, so allocating a more exact slice up-front allows for saving memory when the array list never exceeds that initial allocation size. There are use cases where this precise capacity is useful outside of the `init` function, though, like in instances where the user does not call the `init` function themselves but otherwise knows that an ArrayList is empty so calling `ensureTotalCapacityPrecise` can give the same memory savings that `initCapacity` would have.
Closes#9775
As suggested by @leecannon, this provides more flexibility to the
`Random` interface. For exmaple, this allows for an implementation to
provide multiple different fill functions.
Add an option to allow the '-z notext' option to be passed to the linker
via. the compiler frontend, which is a flag that tells the linker that
relocations in read-only sections are permitted. Certain targets such as
Solana BPF rely on this flag.
Expose all linker options i.e. '-z nodelete', '-z now', '-z relro' in
the compiler frontend. Usage documentation has been updated accordingly.
Expose the '-z notext' flag in the standard library build runner.
Makes `std.meta.trait.hasFn` work as expected for opaque types with function declarations. Alternative is to add clause directly to `std.meta.trait.hasFn` to account for opaque types.
The ensureUnusedCapacity did not reserve a big enough number. I changed
it to no longer guess the capacity because I saw that the number of
possible items was not determinable ahead of time and this can therefore
avoid allocating more memory than necessary.
* test_functions: properly add dependencies of the array on test
functions and test names so that the order comes out correctly.
* fix lowering of struct literals to add parentheses around the type
name.
* omit const qualifier in slices because otherwise slices cannot be
reassigned even when they are local variables.
* special case pointer to functions and double pointer to functions in
renderTypeAndName. This code will need to be cleaned up but for now
it helps us make progress on other C backend stuff.
* fix slice element access to lower to `.ptr[` instead of `[`.
* airSliceElemVal: respect volatile slices
The C backend is the only backend that requires each decl to be output
in an order that satisfies the dependency graph. Here it is implemented
with a simple algorithm based on a `remaining_decls` set, using the
`dependencies` edges that are already stored for each Decl.
This satisfies incremental compilation as well as how `zig test` works,
which calls `updateDecl` on `test_functions`.
Previously when running `zig test` with the C backend, it would return
`error.InvalidExe` because it would try to run the C code source file as
a binary. Now, by default, it will try to run it with
`zig run -lc foo.c` and this can be overridden with the standard
`--test-cmd` flags.
There is a table of `misc_failures` which previously did not allow
multiple errors for the same enum tag. Now it allows this by freeing the
previous compile error and replacing it with the new one. Typically this
will happen because multiple sub-Compilation objects fail with the same
problem, such as not being able to build glibc because of not having
LLVM extensions enabled.
The way `zig test` works is that it uses a stand-in
var test_functions: []const TestFn = undefined;
during semantic analysis, but then just before codegen, it swaps out the
value with a constant like this:
const test_functions: []const TestFn = .{foo, bar, baz, etc};
Before this commit, the `Module.Variable` associated with the stand-in
value was leaked; now it is properly cleaned up before being replaced.
These changes have been made to resolve issue #10037. The `Random`
interface was implemented in such a way that causes significant slowdown
when calling the `fill` function of the rng used.
The `Random` interface is no longer stored in a field of the rng, and is
instead returned by the child function `random()` of the rng. This
avoids the performance issues caused by the interface.
The main problem that motivated these changes is that global constants
which are referenced by pointer would not be emitted into the binary.
This happened because `semaDecl` did not add `codegen_decl` tasks for
global constants, instead relying on the constant values being copied as
necessary. However when the global constants are referenced by pointer,
they need to be sent to the linker to be emitted.
After making global const arrays, structs, and unions get emitted, this
uncovered a latent issue: the anonymous decls that they referenced would
get garbage collected (via `deleteUnusedDecl`) even though they would
later be referenced by the global const.
In order to solve this problem, I introduced `anon_work_queue` which is
the same as `work_queue` except a lower priority. The `codegen_decl`
task for anon decls goes into the `anon_work_queue` ensuring that the
owner decl gets a chance to mark its anon decls as alive before they are
possibly deleted.
This caused a few regressions, which I made the judgement call to add
workarounds for. Two steps forward, one step back, is still progress.
The regressions were:
* Two behavior tests having to do with unions. These tests were
intentionally exercising the LLVM constant value lowering, however,
due to the bug with garbage collection that was fixed in this commit,
the LLVM code was not getting exercised, and union types/values were
not implemented correctly, due to me forgetting that LLVM does not
allow bitcasting aggregate values.
- This is worked around by allowing those 2 test cases to regress,
moving them to the "passing for stage1 only" section.
* The test-stage2 test cases (in test/cases/*) for non-LLVM backends
previously did not have any calls to lower struct values, but now
they do. The code that was there was just `@panic("TODO")`. I
replaced that code with a stub that generates the wrong value. This
is an intentional miscompilation that will obviously need to get
fixed before any struct behavior tests pass. None of the current
tests we have exercise loading any values from these global const
structs, so there is not a problem until we try to improve these
backends.
* Fix bug in exp2_64 to handle negative values (bad translation from C)
* Apply fix to exp2_32() as well, and modify comment on musl behaviour
* Use +%= instead of @addWithOverflow()
Previously, it would emit a ret_ptr AIR instruction but that is not
correct because such an instruction would reference the result pointer
of the caller function rather than the callee function.
Instead, we emit an alloc instruction in this case. `ret_load` already
handles inlining correctly.
* C pointer types always have allowzero set to true but they omit the
word allowzero when printed.
* Implement coercion from C pointers to other pointers.
* Implement in-memory coercion for slices and pointer-like optionals.
* Make slicing a C pointer drop the allowzero bit.
* Value representation for pointer-like optionals is now allowed to use
pointer tag values in addition to the `opt_payload` tag.
Switch prong values are fetched by index in semantic analysis by prong
offset, but these were computed as capture offset. This means that a switch
where the first prong does not capture and the second does, the switch_capture
zir instruction would be assigned switch_prong 0 instead of 1.
* AstGen: always use `typeof` and never `typeof_elem` on the
`switch_cond`/`switch_cond_ref` instruction because both variants
return a value and not a pointer.
- Delete the `typeof_elem` ZIR instruction since it is no longer
needed.
* Sema: validateUnionInit now recognizes a comptime mutable value and
no longer emits a compile error saying "cannot evaluate constant
expression"
- Still to-do is detecting comptime union values in a function that
is not being executed at compile-time.
- This is still to-do for structs too.
* Sema: when emitting a call AIR instruction, call resolveTypeLayout on
all the parameter types as well as the return type.
* `Type.structFieldOffset` now works for unions in addition to structs.
Over the last year of using std.log in practice, it has become clear to
me that having the current 8 distinct log levels does more harm than
good. It is too subjective which level a given message should have which
makes filtering based on log level weaker as not all messages will have
been assigned the log level one might expect.
Instead, more granular filtering should be achieved by leveraging the
logging scope feature. Filtering based on a combination of scope and log
level should be sufficiently powerful for all use-cases.
Note that the self hosted compiler has already limited itself to 4
distinct log levels for many months and implemented granular filtering
based on both log scope and level. This has worked very well in practice
while working on the self hosted compiler.
* without this, when an included relocatable references a common symbol
from another translation unit would not be correctly removed from
the unresolved lookup table triggering a misleading assertion down
the line
* assert upon removal that we indeed removed a ref instead of silently
ignoring in debug
* add test case that covers this issue
According to the documentation, `divTrunc` is "Truncated division.
Rounds toward zero". Lower it as a straightforward fdiv + trunc sequence
to make it behave as expected with mixed positive/negative operands.
Closes#10001
The TLS area may be located in the upper part of the address space and,
if the platform expects a constant offset to be applied, may make the tp
register calculation overflow.
Use +% instead of +, the overflow is harmless.
* Fix backend using wrong union field of the slice instruction.
* LLVM backend properly sets alignment on global variables.
* Sema: add coercion for *T to *[1]T
* Sema: pointers to Decls with explicit alignment now have alignment
metadata in them.
Also switch to the more efficient encoding of the bitcast instruction
when the destination type is anyerror in 2 common cases.
LLVM backend: fix using the wrong type as the optional payload type in
the `wrap_optional` AIR instruction.
After a discussion about language specs, this seems like the best way to
go, because it's simpler to reason about both for humans and compilers.
The `bitcast_result_ptr` ZIR instruction is no longer needed.
This commit also implements writing enums, arrays, and vectors to
virtual memory at compile-time.
This unlocked some more of compiler-rt being able to build, which
in turn unlocks saturating arithmetic behavior tests.
There was also a memory leak in the comptime closure system which is now
fixed.
Each element of the output JSON has the VM address of the generated
binary nondecreasing (some elements might occupy the same VM address
for example the atom and the relocation might coincide in the address
space).
The generated JSON can be inspected manually or via a preview tool
`zig-snapshots` that I am currently working on and will allow the user
to inspect interactively the state of the linker together with the
positioning of sections, symbols, atoms and relocations within each
snapshot state, and in the future, between snapshots too. This should
allow for quicker debugging of the linker which is nontrivial when
run in the incremental mode.
Note that the state will only be dumped if the compiler is built with
`-Dlink-snapshot` flag on, and then the compiler is passed `--debug-link-snapshot`
flag upon compiling a source/project.
AIR:
* div is renamed to div_trunc.
* Add div_float, div_floor, div_exact.
- Implemented in Sema and LLVM codegen. C backend has a stub.
Improvements to std.math.big.Int:
* Add `eqZero` function to `Mutable`.
* Fix incorrect results for `divFloor`.
Compiler-rt:
* Add muloti4 to the stage2 section.
This fixes InstallRawStep to handle the cases when there are empty segments (segments with no sections). Before this change, if there was an empty segment with no sections, then the fixup of binaryOffsets is skipped. This fixes that by looping through each segment until a non-empty one is found and then fixing up the sections. This fixed an issue I was having with InstallRawStep for a bootloader I'm writing.
This modifies the error for an unexpected exit code from the ChildProcess of RunStep to be UnexpectedExitCode rather than UncleanExit. This allows the handler of the error to distinguish between an error reported by the ChildProcess, and an error executing the ChildProcess, which is an important dinstinction when it comes to know what information to report to the user. For example, if you have a ChildProcess that you know reports its own errors, an unexpected exit code would mean the error is already reported, but an unclean exit would mean that child process was not able to report any error.
* Restructure elemPtr a bit
* New AIR instruction: slice_elem_ptr, which returns a pointer to an element of a slice
* Value: adapt elemPtr to work on slices
* New AIR instruction: slice, which constructs a slice out of a pointer
and a length.
* AstGen: use `coerced_ty` for start and end expressions, use `none`
for the sentinel, and don't try to load the result of the slice
operation because it returns a by-value result.
* Sema: pointer arithmetic is extracted into analyzePointerArithmetic
and it is used by the implementation of slice.
- Also I implemented comptime pointer addition.
* Sema: extract logic into analyzeSlicePtr, analyzeSliceLen and use them
inside the slice semantic analysis.
- The approach in stage2 is much cleaner than stage1 because it uses
more granular analysis calls for obtaining the slice pointer, doing
arithmetic on it, and checking if the length is comptime-known.
* Sema: use the slice Value Tag for slices when doing coercion from
pointer-to-array.
* LLVM backend: detect when emitting a GEP instruction into a
pointer-to-array and add the extra index that is required.
* Type: ptrAlignment for c_void returns 0.
* Implement Value.hash and Value.eql for slices.
* Remove accidentally duplicated behavior test.
* AstGen: Move `refToIndex` and `indexToRef` to Zir
* ZIR: the switch_block_*_* instruction tags are collapsed into one
switch_block tag which uses 4 bits for flags, and reduces the
scalar_cases_len field from 32 to 28 bits.
This freed up more ZIR tags, 2 of which are now used for
`switch_cond` and `switch_cond_ref` for producing the switch
condition value. For example, for union values it returns the
corresponding enum value.
* switching with multiple cases and ranges is not yet supported because
I want to change the ZIR encoding to store index pointers into the
extra array rather than storing prong indexes. This will avoid O(N^2)
iteration over prongs.
* AstGen now adds a `switch_cond` on the operand and then passes the
result of that to the `switch_block` instruction.
* Sema: partially implement `switch_capture_*` instructions.
* Sema: `unionToTag` notices if the enum type has only one possible value.
The remaining uses of this result location were causing a bunch of errors
problems where the pointers returned from rvalue and lvalue expressions
would be confused, allowing for extra pointers on rvalue expressions.
For example:
```zig
const X = struct {a: i32};
var x: X = .{.a = 1};
var ptr = &x;
_ = x.a;
```
In the last line, the lookup of x with result location .none_or_ref would
return a double pointer (**X). This would be dereferenced one, after which
a relative pointer to `a` would be fetched and derefenced to get the final
result.
However, this also allows us to manually construct a double pointer, and
fetch the field of the inner type of that:
```zig
_ = &(&(x)).a;
```
This problem also manifests itself with element access. There are two obvious
ways to fix the problem, both of which include replacing the usage of
.none_or_ref for field- and element accesses with something which
deterministically produce either a pointer or value: either result location
.ref or .none. In the former case, this would be paired with .elem_ptr, and
in the latter case with .elem_val.
Note that the stage 1 compiler does not have this problem, because there is
no equivalent of .elem_val and .field_val. In this way it is equivalent to
using the result location .ref for field- and element accesses.
In this case i have used .none, as this matches language behaviour more
closely.
Dereferencing single pointers is now handled outside of the main switch,
which allows deduplication of some cases. This also implements the
relevant operations for pointers to types and pointers to slices.
* do not add linkage scope to aliased exported symbols - this is
not respected on macOS
* special-case `MachO.openPath` in `link.File.openPath` as on macOS
we always link with zld
* redirect to `MachO.flushObject` when linking relocatable objects
in MachO linker whereas move the entire linking logic into
`MachO.flushModule`
When returning an error set or an error union from a function which has
an inferred error set, it populates the error names in addition to the
set of functions. This can have false negatives, meaning that after
checking the map of an unresolved error set, one must do full error set
resolution before emitting a compile error.
* Sema: fix returned operands not coercing to the function return type
in some cases.
- When returning an error or an error union from a function with an
inferred error set, it will now populate the inferred error set.
- Implement error set coercion for the common case of inferred error
set to inferred error set, without forcing a full resolution.
* LLVM backend: update instruction lowering that handles error unions
to respect `isByRef`.
- Also implement `wrap_err_union_err`.
* Relax compile error for "unable to export type foo" to allow
integers, structs, arrays, and floats. This will need to be further
improved to do the same checks as we do for C ABI struct field types.
* LLVM backend: fix extern variables
* LLVM backend: implement AIR instruction `wrap_err_union_payload`
* Sema: implement peer type resolution for optionals and null.
* Rename `Module.optionalType` to `Type.optional`.
* LLVM backend: re-use anonymous values. This is especially useful when
isByRef()=true because it means re-using the same generated LLVM globals.
* LLVM backend: rework the implementation of is_null and is_non_null
AIR instructions. Generate slightly better LLVM code, and also fix
the behavior for optionals whose payload type is 0-bit.
* LLVM backend: improve `cmp` AIR instruction lowering to support
pointer-like optionals.
* `Value`: implement support for equality-checking optionals.
* LLVM backend: The `alloc` AIR instruction as well as pointer
constants which point to a 0-bit element type now call a common
codepath to produce a `*const llvm.Value` which is a non-zero pointer
with a bogus-but-properly-aligned address.
* LLVM backend: improve the lowering of optional types.
* Type: `hasCodeGenBits()` now returns `true` for pointers even when
it returns `false` for their element types.
Effectively, #6706 is now implemented in stage2 but not stage1.
* saturating shl - check for negative rhs at comptime
- adds expected compile_errors case for negative rhs
* add expected compile error for sat shl assign
* std.os: take advantage of `@minimum`. It's probably time to
deprecate `std.min` and `std.max`.
* New AIR instructions: min and max
* Introduce SIMD vector support to stage2
* Add `@Type` support for vectors
* Sema: add `checkSimdBinOp` which can be re-used for other arithmatic
operators that want to support vectors.
* Implement coercion from vectors to arrays.
- In backends this is handled with bitcast for vector to array,
however maybe we want to reduce the amount of branching by
introducing an explicit AIR instruction for it in the future.
* LLVM backend: implement lowering vector types
* Sema: Implement `slice.ptr` at comptime
* Value: improve `numberMin` and `numberMax` to support floats in
addition to integers, and make them behave properly in the presence
of NaN.
* `Module.Union.getFullyQualifiedName` returns a sentinel-terminated
slice so that backends that need null-termination do not need an
additional copy.
* Module.Union: implement a `getLayout` function which returns
information about ABI size and alignment so that the LLVM backend can
properly lower union types into llvm types.
* Sema: `resolveType` now returns `error.GenericPoison` rather than a
Type with tag `generic_poison`. Callsites that want to allow that
need to bypass this higher-level function.
* Sema: implement coercion of enums and enum literals to unions.
* Sema: fix comptime mutation of pointers to unions
* LLVM backend: fully implement proper lowering of union types and
values according to the union layout, and update the handling of AIR
instructions that deal with unions to support union layouts.
* LLVM backend: handle `decl_ref_mut`
- Maybe this should be unreachable since comptime vars should be
changed to be non-mutable when they go out of scope, but it's
harmless for the LLVM backend to support lowering the value.
* Type: fix `requiresComptime` for optionals, pointers, and some other
types. This function is still wrong for structs, unions, and enums.
* Make `alloc` AIR instructions call `resolveTypeLayout`.
* `Sema.zirResolveInferredAlloc` now calls `requireRuntimeBlock` in the
case that it operates on a non-comptime instruction.
* `Type.abiSize` and `Type.abiAlignment` now return 0 for `void`
* Sema: implement `resolveTypeFields` for unions.
* LLVM Backend: support `ptr_elem_ptr` when the element type is 0-bit.
* Type: improve `abiAlignment` implementation for structs to properly
handle fields with non-default alignment.
* Value: implement hashing array, vector, and structs.
The logic for `buildAlloca` had a null deref when the latest alloca was
the last instruction in the entry block. Now the logic is simplified to
always insert alloca instructions first before all other instructions.
There is no longer a need to track `entry_block` or `latest_alloca_inst`;
these fields are deleted frem `FuncGen`.
* apply late symbol resolution for globals - instead of resolving
the exact location of a symbol in locals, globals or undefs,
we postpone the exact resolution until we have a full picture
for relocation resolution.
* fixup stubs to defined symbols - this is currently a hack rather
than a final solution. I'll need to work out the details to make
it more approachable. Currently, we preemptively create a stub
for a lazy bound global and fix up stub offsets in stub helper
routine if the global turns out to be undefined only. This is quite
wasteful in terms of space as we create stub, stub helper and lazy ptr
atoms but don't use them for defined globals.
* change log scope to .link for macho.
* remove redundant code paths from Object and Atom.
* drastically simplify the contents of Relocation struct (i.e., it is
now a simple superset of macho.relocation_info), clean up relocation
parsing and resolution logic.
* ZIR: the `array_type_sentinel` now has a source node attached to it
for proper error reporting.
* Refactor: move `Module.arrayType` to `Type.array`
* Value: the `bytes` and `array` tags now include the sentinel, if the
type has one. This simplifies comptime evaluation logic.
* Sema: fix `zirStructInitEmpty` to properly handle when the type is
void or a sentinel-terminated array. This handles the syntax `void{}`
and `[0:X]T{}`.
* Sema: fix the logic for reporting "cannot store runtime value in
compile time variable" as well as for emitting a runtime store when a
pointer value is comptime known but it is a global variable.
* Sema: implement elemVal for double pointer to array. This can happen
with this code for example: `var a: *[1]u8 = undefined; _ = a[0];`
* Sema: Rework the `storePtrVal` function to properly handle nested
structs and arrays.
- Also it now handles comptime stores through a bitcasted pointer.
When the pointer element type and the type according to the Decl
don't match, the element value is bitcasted before storage.
* Add AIR instructions: ret_ptr, ret_load
- This allows Sema to be blissfully unaware of the backend's decision
to implement by-val/by-ref semantics for struct/union/array types.
Backends can lower these simply as alloc, load, ret instructions,
or they can take advantage of them to use a result pointer.
* Add AIR instruction: array_elem_val
- Allows for better codegen for `Sema.elemVal`.
* Implement calculation of ABI alignment and ABI size for unions.
* Before appending the following AIR instructions to a block,
resolveTypeLayout is called on the type:
- call - return type
- ret - return type
- store_ptr - elem type
* Sema: fix memory leak in `zirArrayInit` and other cleanups to this
function.
* x86_64: implement the full x86_64 C ABI according to the spec
* Type: implement `intInfo` for error sets.
* Type: implement `intTagType` for tagged unions.
The Zig type tag `Fn` is now used exclusively for function bodies.
Function pointers are modeled as `*const T` where `T` is a `Fn` type.
* The `call` AIR instruction now allows a function pointer operand as
well as a function operand.
* Sema now has a coercion from function body to function pointer.
* Function type syntax, e.g. `fn()void`, now returns zig tag type of
Pointer with child Fn, rather than Fn directly.
- I think this should probably be reverted. Will discuss the lang
specs before doing this. Idea being that function pointers would
need to be specified as `*const fn()void` rather than `fn() void`.
LLVM backend:
* Enable calling the panic handler (previously this just
emitted `@breakpoint()` since the backend could not handle the panic
function).
* Implement sret
* Introduce `isByRef` and implement it for structs and arrays. Types
that are `isByRef` are now passed as pointers to functions, and e.g.
`elem_val` will return a pointer instead of doing a load.
* Move the function type creating code from `resolveLlvmFunction` to
`llvmType` where it belongs; now there is only 1 instance of this
logic instead of two.
* Add the `nonnull` attribute to non-optional pointer parameters.
* Fix `resolveGlobalDecl` not using fully-qualified names and not using
the `decl_map`.
* Implement `genTypedValue` for pointer-like optionals.
* Fix memory leak when lowering `block` instruction and OOM occurs.
* Implement volatile checks where relevant.
isZigPrimitiveType had a bug where it checked the integer names (e.g.
u32) before primitives, leading it to incorrectly return `false` for
`undefined` which starts with `u`.
Related: #9928
New ZIR instruction: elem_ptr_imm
This saves some memory for array literals since the element indexes are
communicated as immediate values rather than as references to other ZIR
instructions.
* Sema: fix a missing copy on enum tag values
* LLVM backend: fix lowering of enum constant values for enums with
specified tag values.
* Value: fix enumToInt for `enum_numbered` cases.
The float widening behavior tests which rely on compiler-rt symbols are
now passing.
* Update AIR instruction `intcast` to allow the dest type to be an
enum.
* LLVM backend: update `intcast` to support when the bit counts of
operand and dest type are the same. This was already a requirement of
the instruction previously.
* Type: `intInfo` supports the case when the type is an enum, and
retrieves the info for the integer tag type. This makes it pretty
easy for backends to implement `intcast` without having to care
explicitly that the new type is an enum. As a bonus, simple enums
never have to go through the type system; their signedness and bit
count are computed directly.
The "int to enum" behavior test case is now passing for stage2 in the
LLVM backend.
* work around a stage1 miscompilation leading to the wrong integer
comparison predicate being emitted.
* fix the bug of not annotating callsites with the calling convention
of the callee, leading to undefined behavior.
* add the `nobuiltin` attribute when building freestanding libc or
compiler_rt libraries to prevent e.g. memcpy from being "optimized"
into a call to itself.
* compiler-rt: change a call to be comptime to make the generated LLVM
IR simpler and easier to study.
I still can't enable the widening tests due to the compiler-rt compare
function being miscompiled in some not-yet-diagnosed way.
libc requires this to use `long double` which is sometimes the same as
f128, sometimes not.
Also for an unknown reason, aarch64 is getting an invalid result for the
`@mulAdd` behavior test for f128. See #9900.
* move fmaq from freestanding libc to compiler_rt, unconditionally
exported weak_odr.
* stage1: add fmaf, fmal, fmaq as symbols that compiler-rt might
generate calls to.
* stage1: lower `@mulAdd` directly to a call to `fmaq` instead of to
the LLVM intrinsic because LLVM will lower it to `fmal` even when the
target's `long double` is not equivalent to `f128`.
This commit is intended to fix the test suite which is failing on the
previous commit.
std: add f128 implementations of fma, frexp, and ilogb. Expose `fmal` in
zig's freestanding libc. This makes `@mulAdd` work correctly for f128.
Fixes a CI regression from yesterday, where I added a usage of f128
`@mulAdd` into the self-hosted compiler.
Before, Sema for comptime `@bitCast` would return the same Value but
change the Type. This gave invalid results because, for example, an
integer Value when the Type is a float would be interpreted numerically,
but `@bitCast` needs it to reinterpret how they would be stored in
memory.
This requires a mechanism to serialize a Value to a byte buffer and
deserialize a Value from a byte buffer.
Not done yet, but needs to happen: comptime dereferencing a pointer
to a Decl needs to perform a comptime bitcast on the loaded value.
Currently the value is silently wrong in the same way that `@bitCast`
was silently wrong before this commit.
The logic in Value for handling readFromMemory for large integers is
only correct for small integers. It needs to be fleshed out for proper
big integers.
As part of this change:
* std.math.big.Int: initial implementations of readTwosComplement and
writeTwosComplement. They only support bit_count <= 128 so far and
panic otherwise.
* compiler-rt: move the compareXf2 exports over to the stage2 section.
Even with the improvements in this commit, I'm still seeing test
failures in the widening behavior tests; more investigation is
needed.
Upstream LLVM fixes#8597, no longer need `-Dskip-debug` and others.
Additionally, due to the nature of drone.io server pool, it is
beneficial to know which aarch64 CPU brand is in use.
- drop `-Dskip-debug` and others
- invoke `lscpu` prior to build
- enable more testsuite consistent with ci azure
- remove workaround for (already closed) #6830closes#8597
Updates the Language Reference sections: Comments, Values, and Zig Test.
Zig Test section moved down with the goal "make sure it can be read top to
bottom sensibly" in mind (issue #1524).
Comments and Values section examples changed test declarations to a main
function and expect statement to print statements.
A print statement was added to the "String Literals and Unicode Code Point"
section's example to demonstrate the "u" format specifier.
Zig Test Section:
* Addresses the question: "How does the syntax work?".
* Partially answers the question: "What can I do with the zig test tool?" but
should be sufficient to understand the examples in all of this document.
* Addresses the question: "How does a top-level test block differ from a function definition?"
* Provides a example to run multiple test.
Lacks clear definitions of containers, top-level, order independence, lazy
analysis, resolve, reference.
GitHub Issues: #8221, #8234
This was an attempt to move saturating_arithmetic.zig to the "passing
for stage2" section, which did not pan out due to the discovery of 2
prerequisite items that need to be done, but I did make a bug fix along
the way of the calculation of max/min integers.
This commit also simplifies the saturating arithmetic behavior tests to
depend on less of the zig language that is not related to saturating
arithmetic.
This function can be used to initialize a big integer to either the upper
or lower limit of a 2s-complement integer. Note that the result is still
in sign-magnitude representation, though in order to convert it into twos
complement all one has to do is take the absolute value.
lladd is now implemented in terms of lladdcarry, which returns the carry limb.
Similarly, llsub is implemented using llsubcarry, which returns the borrow limb.
Decl names are now rendered fully qualified, so that e.g. "main" in two
different files does not conflict with each other.
Additionally, they are now mangled so as to not generate invalid C
identifier names.
Before, the system to replace a result location pointer with a
traditional break instruction did not notice the case when one of the
bodies was unreachable. Now, the emitted ZIR code is improved and
simplified in this case, resulting in a new passing behavior test.
tar does not properly implement the -k feature, so don't use it.
It incorrectly reports "file exists" when the expected behavior is to
leave the existing file alone.
NetBSD CI is disabled because it is not yet supported in
zig-bootstrap. Once NetBSD has proper zig-bootstrap support, it can be
re-enabled.
Windows is not solved here yet; will be pushing a separate commit for
that.
These functions are intended to emit compile errors when Clang adds new
items to its enums. However, two of them were casting to the Zig enum
and switching on that, which wouldn't detect anything useful.
* stage2: array mul support more types of operands
* stage2: array cat support more types of operands
* print_zir: print array_init
* stage2: implement Sema for array_init
There was duplicated logic for whether to include compiler_rt in the
linker line both in the frontend and in the linker backends. Now the
logic is only in the frontend; the linker puts it on the linker line if
the frontend provides it.
Fixes the CI failures.
This prevents a compiler_rt built with stage2 (which is intentionally
different than when built with stage1) from being used for stage1 and
vice versa.
Fixes the regression from the previous commit.
* AstGen: fix emitting `store_to_inferred_ptr` when it should be emitting
`store` for a variable that has an explicit alignment.
* Compilation: fix a couple memory leaks
* Sema: implement support for locals that have specified alignment.
* Sema: implement `@intCast` when it needs to emit an AIR instruction.
* Sema: implement `@alignOf`
* Implement debug printing for extended alloc ZIR instructions.
Also improve the LLVM backend to support lowering bigints to LLVM
values.
Moves over a bunch of math.zig test cases to the "passing for stage2"
section.
* `@as` and `@bitCast` no longer unconditionally return `true` from
this function; they forward the question to their sub-expression.
* fix `@splat` incorrectly being marked as needing a memory location
(this function returns a SIMD vector; it definitely does not want a
memory location).
Makes AstGen generate slightly nicer ZIR, which in turn generates
slightly nicer AIR, generating slightly nicer machine code in debug
builds.
It also means I can procrastinate implementing the bitcast_result_ptr
ZIR instruction semantic analysis :^)
* Remove the builtins `@addWithSaturation`, `@subWithSaturation`,
`@mulWithSaturation`, and `@shlWithSaturation` now that we have
first-class syntax for saturating arithmetic.
* langref: Clarify the behavior of `@shlExact`.
* Ast: rename `bit_shift_left` to `shl` and `bit_shift_right` to `shr`
for consistency.
* Air: rename to include underscore separator with consistency with
the rest of the ops.
* Air: add shl_exact instruction
* Use non-extended tags for saturating arithmetic, to keep it
simple so that all the arithmetic operations can be done the same
way.
- Sema: unify analyzeArithmetic with analyzeSatArithmetic
- implement comptime `+|`, `-|`, and `*|`
- allow float operands to saturating arithmetic
* `<<|` allows any integer type for the RHS.
* C backend: fix rebase conflicts
* LLVM backend: reduce the amount of branching for arithmetic ops
* zig.h: fix magic number not matching actual size of C integer types
- modify AstGen binOpExt()/assignBinOpExt() to accept generic extended payload T
- rework Sema zirSatArithmetic() to use existing sema.analyzeArithmetic() by adding an `opt_extended` parameter.
- add airSatOp() to codegen/c.zig
- add saturating functions to src/link/C/zig.h
- adds initial support for the operators +|, -|, *|, <<|, +|=, -|=, *|=, <<|=
- uses operators in addition to builtins in behavior test
- adds binOpExt() and assignBinOpExt() to AstGen.zig. these need to be audited
* AIR: add `mod` instruction for modulus division
- Implement for LLVM backend
* Sema: implement `@mod`, `@rem`, and `%`.
* Sema: fix comptime switch evaluation
* Sema: implement comptime shift left
* Sema: fix the logic inside analyzeArithmetic to handle all the
nuances between the different mathematical operations.
- Implement comptime wrapping operations
* Merge call zir instructions to make space for field_call
* Fix bug with comptime known anytype args
* Delete the param_type zir instruction
* Move some passing tests to stage 2
* Implement a.b() function calls
* Add field_call_bind support for call and field builtins
* AIR: add `get_union_tag` instruction
- implement in LLVM backend
* Sema: implement == and != for union and enum literal
- Also implement coercion from union to its own tag type
* Value: implement hashing for union values
The motivating example is this snippet:
comptime assert(@typeInfo(T) == .Float);
This was the next blocker for stage2 building compiler-rt.
Now it is switch at compile-time on an integer.
* AIR instructions struct_field_ptr and related functions now are also
emitted by the frontend for unions. Backends must inspect the type
of the pointer operand to lower the instructions correctly.
- These will be renamed to `agg_field_ptr` (short for "aggregate") in
the future.
* Introduce the new `set_union_tag` AIR instruction.
* Introduce `Module.EnumNumbered` and associated `Type` methods. This
is for enums which have no decls, but do have the possibility of
overriding the integer tag type and tag values.
* Sema: Implement support for union tag types in both the
auto-generated and explicitly-provided cases, as well as explicitly
provided enum tag values in union declarations.
* LLVM backend: implement lowering union types, union field pointer
instructions, and the new `set_union_tag` instruction.
docgen HTML escapes characters inside of `syntax_block`s. This commit replaces the escaped
greater than with the `>` character. No other occurrences were found.
Fixes#9840
On certain systems (Solaris), resolving the scope id from an interface
name can only be done on AF_INET-domain sockets. While we're here,
simplify the test while we're here, since there's only one address.
Also note that the loopback interface name is not stable across OSs.
BSDs and Solaris use `lo0` whilst Linux uses `l0`.
* LLVM backend: respect `sub_path` just like the other stage2 backends
do.
* Compilation has some new logic to only emit work queue jobs for
building stuff when it believes itself to be capable. The linker
backends no longer have duplicate logic; instead they respect the
optional bit on the respective asset.
Introduce an explicit decl_map for *Decl to LLVMValueRef. Doc comment
reproduced here:
Ideally we would use `llvm_module.getNamedFunction` to go from *Decl to
LLVM function, but that has some downsides:
* we have to compute the fully qualified name every time we want to do the lookup
* for externally linked functions, the name is not fully qualified, but when
a Decl goes from exported to not exported and vice-versa, we would use the wrong
version of the name and incorrectly get function not found in the llvm module.
* it works for functions not all globals.
Therefore, this table keeps track of the mapping.
Non-exported functions now use fully-qualified symbol names.
`Module.Decl.getFullyQualifiedName` now returns a sentinel-terminated
slice which is useful to pass to LLVMAddFunction.
Instead of using aliases for all external symbols, now the LLVM backend
takes advantage of LLVMSetValueName to rename functions that become
exported. Aliases are still used for the second and remaining exports.
freeDecl is now handled properly in the LLVM backend, deleting the
LLVMValueRef corresponding to the Decl being deleted. The linker
backends for ELF, COFF, Mach-O, and Wasm had to be updated to forward
the freeDecl call to the LLVM backend.
Extracts lib/std/special/c_stage1.zig from lib/std/special/c.zig.
When the self-hosted compiler is further along, all the logic from c_stage1.zig will
be migrated back c.zig and then c_stage1.zig will be deleted. Until then we have a
simpler implementation of c.zig that only uses features already implemented in self-hosted.
So far it only contains memcpy and memset, with slightly different
(arguably more correct!) implementations that are compatible with
self-hosted.
Additionally, this commit improves the LLVM backend:
* use the more efficient and convenient fnInfo() when lowering function
type info.
* fix incremental compilation not deleting all basic blocks of a
function.
* hook up calling conventions
* hook up the following function attributes:
- noredzone, nounwind, uwtable, minsize, optsize, sanitize_thread
* AstGen: fix not emitting `struct_init_empty` when an explicit type is
present in struct initialization syntax.
* AstGen: these two syntaxes now lower to identical ZIR:
- `var a = A{ .b = c };`
- `var a = @as(A, .{ .b = c });`
* Zir: clarify `auto_enum_tag` in the doc comments.
* LLVM Backend: fix lowering of function return types when the type has
0 bits.
* `Type.hasCodeGenBits` this function is used to find out if it ever
got sent to a linker backend for lowering. In the case that a struct
never has its struct fields resolved, this will be false. In such a
case, no corresponding `freeDecl` needs to be issued to the linker
backend. So instead of asserting the fields of a struct are resolved,
this function now returns `false` for this case.
* `Module.clearDecl` there was logic that asserted when there is no
outdated_decls map, any dependants of a Decl being cleared had to be
in the deletion set. However there is a possible scenario where the
dependant is not in the deletion set *yet* because there is a Decl
which depends on it, about to be deleted. If it were added to an
outdated_decls map, it would be subsequently removed from the map
when it gets deleted recursively through its dependency being
deleted.
These issues were uncovered via unrelated changes which are the two
commits immediately preceding this one.
* prepare compiler-rt to support being compiled by stage2
- put in a few minor workarounds that will be removed later, such as
using `builtin.stage2_arch` rather than `builtin.cpu.arch`.
- only try to export a few symbols for now - we'll move more symbols
over to the "working in stage2" section as they become functional
and gain test coverage.
- use `inline fn` at function declarations rather than `@call` with an
always_inline modifier at the callsites, to avoid depending on the
anonymous array literal syntax language feature (for now).
* AIR: replace floatcast instruction with fptrunc and fpext for
shortening and widening floating point values, respectively.
* Introduce a new ZIR instruction, `export_value`, which implements
`@export` for the case when the thing to be exported is a local
comptime value that points to a function.
- AstGen: fix `@export` not properly reporting ambiguous decl
references.
* Sema: handle ExportOptions linkage. The value is now available to all
backends.
- Implement setting global linkage as appropriate in the LLVM
backend. I did not yet inspect the LLVM IR, so this still needs to
be audited. There is already a pending task to make sure the alias
stuff is working as intended, and this is related.
- Sema almost handles section, just a tiny bit more code is needed in
`resolveExportOptions`.
* Sema: implement float widening and shortening for both `@floatCast`
and float coercion.
- Implement the LLVM backend code for this as well.
Currently -Dcpu is completely ignored if -Dtarget isn't passed as well.
Further, -Dcpu=baseline is ignored even if -Dtarget=native is passed.
This patch fixes these 2 issues, always respecting the -Dcpu option if
present.
Previously, linker backends or machine code backends were able to hold
on to references to inside Sema's temporary arena. However there can
be large objects stored there that we want to free after machine code is
generated.
The primary change in this commit is to use a temporary arena for Sema
of function bodies that gets freed after machine code backend finishes
handling `updateFunc` (at the same time that Air and Liveness get freed).
The other changes in this commit are fixing issues that fell out from
the primary change.
* The C linker backend is rewritten to handle updateDecl and updateFunc
separately. Also, all Decl updates get access to typedefs and
fwd_decls, not only functions.
* The C linker backend is updated to the new API that does not depend
on allocateDeclIndexes and does not have to handle garbage collected
decls.
* The C linker backend uses an arena for Type/Value objects that
`typedefs` references. These can be garbage collected every so often
after flush(), however that garbage collection code is not
implemented at this time. It will be pretty simple, just allocate a
new arena, copy all the Type objects to it, update the keys of the
hash map, free the old arena.
* Sema: fix a handful of instances of not copying Type/Value objects
from the temporary arena into the appropriate Decl arena.
* Type: fix some function types not reporting hasCodeGenBits()
correctly.
Otherwise, for last sections in segments it could happen we would
not expand the segment when actually required thus exceeding the
segment's size and causing data clobbering and dyld runtime errors.
There were two things to resolve here:
* Snektron's branch edited Zir printing, but in master branch
I moved the printing code from Zir.zig to print_zir.zig. So that
just had to be moved over.
* In master branch I fleshed out coerceInMemory a bit more, which
caused one of Snektron's test cases to fail, so I had to add
addrspace awareness to that. Once I did that the tests passed again.
In a previous commit (f4d3d29), syntax checking for code blocks with the
`syntax` type was disabled due to a change in astgen now checking the existence of
identifiers. The change in astgen caused some code samples in the language
reference to cause compilation errors.
This commit updates the code samples in the language reference and
re-enables syntax checking. Some code samples have been changed to unchecked
syntax blocks using `{#syntax_block#}` when suitable.
* introduce float_to_int and int_to_float AIR instructionts and
implement for the LLVM backend and C backend.
* Sema: implement `zirIntToFloat`.
* Sema: implement `@atomicRmw` comptime evaluation
- introduce `storePtrVal` for when one needs to store a Value to a
pointer which is a Value, and assert it happens at comptime.
* Value: introduce new functionality:
- intToFloat
- numberAddWrap
- numberSubWrap
- numberMax
- numberMin
- bitwiseAnd
- bitwiseNand (not implemented yet)
- bitwiseOr
- bitwiseXor
* Sema: hook up `zirBitwise` to the new Value bitwise implementations
* Type: rename `isFloat` to `isRuntimeFloat` because it returns `false`
for `comptime_float`.
This edit allows the reader to understand the syntax this section is talking about more quickly – they don’t have to read the whole code block and understand which part of it demonstrates the feature being described.
Affects https://ziglang.org/documentation/master/#Inferred-Error-Sets
The WebAssembly spec requires signed LEB128 to be encoded up to a maximum number of bytes (max 5 bytes for i32, max 10 bytes for i64) and that "unused" bits are all 0 if the number is positive and all 1 if the number is negative. The Zig LEB128 implementation already enforces the max number of bytes and does check the unused bytes https://github.com/ziglang/zig/blob/master/lib/std/leb128.zig#L70-L79.
However, the WebAssembly test suite has a number of tests that were failing validation (expecting the wasm module to fail validation, but when running the tests, those examples were actually passing validation):
https://github.com/malcolmstill/foxwren/blob/master/test/testsuite/binary-leb128.wast#L893-L902https://github.com/malcolmstill/foxwren/blob/master/test/testsuite/binary-leb128.wast#L934-L943
Notably the failures are both cases of negative numbers and the top 4 bits of the last byte are zero. And I believe this is the issue: we're only currently checking the "unused" / remaining_bits if we overflow, but in the case of 0x0_ no overflow happens and so the bits go unchecked.
In other words:
\xff\xff\xff\xff\7f rightly successfully decodes (because it overflows and the remaining bits are 0b1111)
\xff\xff\xff\xff\6f rightly errors with overflow (because it overflows and the remaining bits are 0b1110)
\xff\xff\xff\xff\0f incorrectly decodes when it should error (because the top 4 bits are all 0, and so no overflow occurs and no check that the unused bits are 1 happens)
This PR adds a the remaining_bits check in an else branch of the @shlWithOverflow when we're looking at the last byte and the number being decoded is negative.
Note: this means a couple of the test cases in leb128.zig that are down as decoding shouldn't actually decode so I added the appropriate 1 bits.
This is a property which solely belongs to pointers to functions,
not to the functions themselves. This cannot be properly represented by
stage 2 at the moment, as type with zigTypeTag() == .Fn is overloaded for
for function pointers and function prototypes.
Validity checks are also based on context; whether the entity being validated
is a mutable/constant value, a pointer (that is ascripted with an addrspace
attribute) or a function with an addrspace attribute. Error messages are
relatively simple for now.
Adds AST generation for address spaces on pointers, function prototypes,
function declarations and variable declarations. In the latter two cases,
declaration properties were already stored more efficiently in a declaration
structure. To accomodate these for address spaces, the bit indicating presence
of a linksection attribute has been extended to include either linksection,
address space, or both.
I believe these are Linux specific so they will need to be os-gated
in `elf.zig` at some point, but I reckon it should be fine to have
them as-is right now since the ELF linker work will mainly be done
on x86-64 Linux at first.
* test runner is improved to respect `error.SkipZigTest`
* start code is improved to `@setAlignStack(16)` before calling main()
* the newly passing behavior test has a workaround for the fact that
stage2 cannot yet call `std.Target.x86.featureSetHas()` at comptime.
This is blocking on comptime closures. The workaround is that there
is a new decl `@import("builtin").stage2_x86_cx16` which is a `bool`.
* Implement `@setAlignStack`. This language feature should be re-evaluated
at some point - I'll file an issue for it.
* LLVM backend: apply/remove the cold attribute and noinline attribute
where appropriate.
* LLVM backend: loads and stores are properly annotated with alignment
and volatile attributes.
* LLVM backend: allocas are properly annotated with alignment.
* Type: fix integers reporting wrong alignment for 256-bit integers and
beyond. Once you get to 16 byte aligned, there is no further
alignment for larger integers.
This supports the case when it is known that LLVM, Clang, LLD were built
with Clang (or `zig c++`). This commit updates the Linux CI script to
pass this since we build using a zig tarball.
Investigating hexops/zorex#4, I found that `--test-evented-io` is currently broken in
the latest Zig nightly. See #9779 for a small reproduction.
The issue is that allocation errors here are not correctly handled, as this function
returns `void` and all other error cases `@panic`, the allocation failure should also
use `@panic`.
Fixes#9779
Helps hexops/zorex#4
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
* In watch mode, when changing the C source, we will trigger complete
relinking of objects, dylibs and archives (atoms coming from the
incremental updates stay put however). This means, we need to undo
metadata populated when linking in objects, archives and dylibs.
* Remove unused splitting section into atoms bit. This optimisation
will probably be best rewritten from scratch once self-hosted
matures so parking the idea for now. Also, for easier management
of atoms spawned from the Object file, keep the atoms subgraph as
part of the Object file struct.
* Remove obsolete ref to static initializers in object struct.
* Implement handling of global symbol collision in updateDeclExports.
Some systems (Solaris, OpenBSD, AIX) change their definitions of
sockaddr_storage to be larger than 128 bytes. This comment adds a new
constant in the `sockaddr` that defines the size for every system.
Fixes#9759
The entire 'path' array would get written to the formatting function,
when it should instead be treated as a regular zero-terminated string.
Note that this doesn't handle abstract paths on Linux, those paths
*start* with a \0 byte and are hence treated as empty strings instead.
But fixing that would require more adjustments than just formatting, in
particular to getOsSockLen().
* langref: add some more "see also" links for atomics
* Add the following AIR instructions
- atomic_load
- atomic_store_unordered
- atomic_store_monotonic
- atomic_store_release
- atomic_store_seq_cst
- atomic_rmw
* Implement those AIR instructions in LLVM and C backends.
* AstGen: make the `ty` result locations for `@atomicRmw`, `@atomicLoad`,
and `@atomicStore` be `coerced_ty` to avoid unnecessary ZIR
instructions when Sema will be doing the coercions redundantly.
* Sema for `@atomicLoad` and `@atomicRmw` is done, however Sema for
`@atomicStore` is not yet implemented.
- comptime eval for `@atomicRmw` is not yet implemented.
* Sema: flesh out `coerceInMemoryAllowed` a little bit more. It can now
handle pointers.
Conflicts:
* cmake/Findclang.cmake
* cmake/Findlld.cmake
* cmake/Findllvm.cmake
In master branch, more search paths were added to these files with "12"
in the path. In this commit I updated them to "13".
* src/stage1/codegen.cpp
* src/zig_llvm.cpp
* src/zig_llvm.h
In master branch, ZigLLVMBuildCmpXchg is improved to add
`is_single_threaded`. However, the LLVM 13 C API has this already, and
in the llvm13 branch, ZigLLVMBuildCmpXchg is deleted in favor of the C
API. In this commit I updated stage2 to use the LLVM 13 C API rather
than depending on an improved ZigLLVMBuildCmpXchg.
Additionally, src/target.zig largestAtomicBits needed to be updated to
include the new m68k ISA.
The following is an azure failure that occured Sep 13:
del : Cannot remove item D:\a\1\s\sfx.exe: The process cannot access the file 'D:\a\1\s\sfx.exe' because it is being used by another process.
Windows will keep a hold of recently run exeutables even after their process has exited. To avoid this I've just removed the deletion of the exe file. It's about 70 MB so it's probably OK.
* Added fseeki64.c from mingw-w64 9.0.0. This file was missing in Zig distribution. This file contains implementation for _fseeki64 and _ftelli64 functions.
* Implement Sema for `@cmpxchgWeak` and `@cmpxchgStrong`. Both runtime
and comptime codepaths are implement.
* Implement Codegen for LLVM backend and C backend.
* Add LazySrcLoc.node_offset_builtin_call_argX 3...5
* Sema: rework comptime control flow.
- `error.ComptimeReturn` is used to signal that a comptime function
call has returned a result (stored in the Inlining struct).
`analyzeCall` notices this and handles the result.
- The ZIR instructions `break_inline`, `block_inline`,
`condbr_inline` are now redundant and can be deleted. `break`,
`block`, and `condbr` function equivalently inside a comptime scope.
- The ZIR instructions `loop` and `repeat` also are modified to
directly perform comptime control flow inside a comptime scope,
skipping an unnecessary mechanism for analysis of runtime code.
This makes Zig perform closer to an interpreter when evaluating
comptime code.
* Sema: zirRetErrValue looks at Sema.ret_fn_ty rather than sema.func
for adding to the inferred error set. This fixes a bug for
inlined/comptime function calls.
* Implement ZIR printing for cmpxchg.
* stage1: make cmpxchg respect --single-threaded
- Our LLVM C++ API wrapper failed to expose this boolean flag before.
* Fix AIR printing for struct fields showing incorrect liveness data.
* add functions to decode an epoch timestamp
The code added here is alternative to the libc gmtime function. This function takes a unix epoch timestamp and decodes it into things like the year/day/time/etc. I looked at various libc implementations to see how it was implemented and this appears to be correct. I reorganized it so that applications can choose which data they need rather than calcualting it all in a single function. The data structures layout the order of operations required to decode various things like the year/month or time of day.
* set Month.jan to 1 instead of 0 to avoid +1 in conversion to numeric
* add another test
* remove unnecessary comptimeMod
This applies to stage2 where we make use of the cache system to work
out if we need to relink objects when performing incremental updates.
When the process is restarted however, while in principle the idea is
to carry on where we left off by reparsing the prelinked binary from
file, the required machinery is not there yet, and therefore we always
fully relink upon restart.
Extract existing constants to do with TCP socket options into a 'TCP'
namespace.
Export 'MSG' and 'TCP' from std.os.{linux, windows} into std.c.
Fix compile errors to do with std.x.os.Socket methods related to setting
TCP socket options.
Handle errors in the case that an interface could not be resolved in an
IPv6 address on Windows. Tested using Wine with the loopback interface
disabled.
Have all instantiations of std.x.os.Socket on Windows instantiate an
overlapped socket descriptor. Fixes the '1ms read timeout' test in
std.x.net.tcp.Client. The test would previously deadlock, as read
timeouts only apply to overlapped sockets.
Windows documentation by default recommends that most instantiations of
sockets on Windows be overlapped sockets (s.t. they may operate in both
blocking or nonblocking mode when operated with WSA* syscalls). Refer to
the documentation for WSASocketA for more info.
The language reference's HTML has been updated to be more semantically correct.
This also helps to improve the document's accessibility concerns.
* Document structure has single h1, other header sections start at h2, nav sections w/ aria labels, main section
* Zig's homepage is linked, Zig Standard Library section link to it
* Tables have caption and scoping rows and columns
* Code blocks are figures with figure captions citing source files
* Change line height 1.5 to include table of contents as well
* Luminosity contrast ratios have been adjusted to 7:1
* Dark mode colors adjusted to reduce eye strain
* Links have default browser underline with hover and focus effects
* Asides, definition lists, keyboard inputs, program outputs are represented semantically
Tools used to check:
WAVE plugin https://wave.webaim.org/
Firefox Accessibility Developer Tool
Lighthouse Accessibility Tool
- On Intel Macs, the path is /usr/local/opt/llvm@12
- On Silicon Macs, the path is /opt/homebrew/opt/llvm@12
This makes specifying CMAKE_PREFIX_PATH optional for Homebrew LLVM.
This is particularly relevant for x86_64 and C++ when relocating
StaticInit sections containing static initializers machine code.
Then, in case of SIGNED_X relocations, it is necessary to have the
full image of the VM address layout of the sections in the object
file as this is how the addend needs to be adjusted for non-extern
relocations.
OpenBSD doesn't implement EVFILT_USER filter for kqueue(2), so we couldn't use that for event loop.
instead, use a EVFILT_TIMER filter with EV_ONESHOT (trigger only once) and delay 0sec (which trigger immediatly).
it fits the usage of EVFILT_USER which is only used to "wakeup" the kevent(2) call from userland.
Also, calculate non-extern, section offset based addends for SIGNED
and UNSIGNED relocations on x86_64 upfront as an offset wrt to the
target symbol representing position of the section/atom within the
final artifact.
Otherwise, Apple's tooling goes mental and reports that the executable
is malformed/fails strict validation. We absolutely have to get it
right to support tools such `codesign` which are required to
successfully launch an app on an iOS device for instance. When Zig
matures enough so that we can ditch any Apple tooling and still be
able to successfully codesign for iOS and other, we can revisit this
area. Until then however, we are stuck in having to rewrite the LINKEDIT
segment at every update run of the self-hosted.
FYI, the strict layout for the MachO binary apparently is (please,
read this with a pinch of salt as this is inferred by me):
* __TEXT segment
* __DATA_CONST segment
* __DATA segment
* __LINKEDIT segment
* dyld info (rebase, bind, weak bind, lazy bind, export)
* symbol table
* dynamic symbol table
* string table
* code signature (if expected)
I incorrectly assumed that __kernel_timespec was used when not linking
libc, however that is not the case. `std.os.timespec` is used both for
libc and non-libc cases. `__kernel_timespec` is a special struct that is
used only for io_uring.
Tests with no names are executed when using `zig test` regardless of the
`--test-filter` used. Non-named tests should be used when simply
importing unit tests from another file. This allows `zig test` to find
all the appropriate tests, even when using `--test-filter`.
* std lib tests are passing on x86_64-linux with and without -lc
* stage2 is building from source on x86_64-linux
* down to 38 remaining uses of `usingnamespace`
The main purpose of this branch is to explore avoiding the
`usingnamespace` feature of the zig language, specifically with regards
to `std.os` and related functionality.
If this experiment is successful, it will provide a data point on
whether or not it would be practical to entirely remove `usingnamespace`
from the language.
In this commit, `usingnamespace` has been completely eliminated from
the Linux x86_64 compilation path, aside from io_uring.
The behavior tests pass, however that's as far as this branch goes. It is
very breaking, and a lot more work is needed before it could be
considered mergeable. I wanted to put a pull requset up early so that
zig programmers have time to provide feedback.
This is progress towards closing #6600 since it clarifies where the
actual "owner" of each declaration is, and reduces the number of
different ways to import the same declarations.
One of the main organizational strategies used here is to do namespacing
with real namespaces (e.g. structs) rather than by having declarations
share a common prefix (the C strategy). It's no coincidence that
`usingnamespace` has similar semantics to `#include` and becomes much
less necessary when using proper namespaces.
This allows writing HEX files with `exe.installRaw`, where `exe` is a
`LibExeObjStep`. A HEX file will be written if the file extension is `.hex`
or `.ihex`, otherwise a binfile will be written. The output format can be
explicitly chosen with `exe.installRawWithFormat("filename", .hex);`
(or `.bin`)
Part of #2826
Co-authored-by: Akbar Dhanaliwala <akbar.dhanaliwala@gmail.com>
scanDecls() made an incorrect assumption about all declarations having
function names. The compile error for "missing function name" needed to
go into scanDecls().
scanDecls() made an incorrect assumption about all declarations having
function names. The compile error for "missing function name" needed to
go into scanDecls().
- adds 1 simple behavior tests for each
which does integer and vector ops at
runtime and comptime
- adds bigint_*_sat() methods for each
- use CreateIntrinsic() which accepts a
variable number of arguments to pass
the scale parameter
* update langref
- added case to test/compile_errors.zig given floats
- explain upstream bug in llvm.smul.fix.sat and link to #9643 in langref and commented out test cases
* sat-arithmetic: skip mul tests if arch == .wasm32 because ci is erroring with 'LLVM ERROR: Unable to expand fixed point multiplication' when compiling for wasm32
Instead of checking for stage1 at every callsite, move the logic
inside `allocateAtom`. This is fine since this logic will disappear
anyhow once I add expanding and shifting segments and sections.
There was some new code in master branch enumerating all the targets and
a new target was added so we needed to add the glue code.
This commit also introduces some build options to support experimental
LLVM targets.
This commit makes it possible to combine self-hosted with a pre-compiled
C object file, e.g.:
```
zig-out/bin/zig build-exe hello.zig add.o
```
where `add.o` is a pre-compiled C object file.
Also:
* improve the "ambiguous reference" error by swapping the order of
"declared here" and "also declared here" notes.
* improve the "not accessible from inner function" error:
- point out that it has to do with the thing being mutable
- eliminate the incorrect association with it being a function
- note where it crosses a namespace boundary
* struct field types are evaluated in a context that has the struct
namespace visible. Likewise with align expressions, linksection
expressions, enum tag values, and union/enum tag argument
expressions.
Closes#9194Closes#9622
Conflicts:
lib/libcxx/include/__config
d57c0cc3bf added support for DragonFlyBSD
to libc++ by updating some ifdefs. This needed to be synced with llvm13.
* stage2 AstGen: add missing compile error for declaring a local
that shadows a primitive. Even with `@""` syntax, it may not have
the same name as a primitive.
* stage2 AstGen: add a compile error for a global declaration
whose name matches a primitive. However it is allowed when using
`@""` syntax.
* stage1: delete all "declaration shadows primitive" compile errors
because they are now handled by stage2 AstGen.
* stage1/stage2 AstGen: notice when using `@""` syntax and:
- treat `_` as a regular identifier
- skip checking if an identifire is a primitive
Check the new test cases for clarifications on semantics.
closes#6062
This way, we will conform to the standard practice of setting the
offset within the section header to the beginning of the file and
we will be able to track the location of the section in the file
for incremental updates.
* rework `build_options` to integrate with the FileSource abstraction
* support mapping as an arbitrarily named package
* support mapping to multiple different artifacts
* use hash of contents for options filename
Locals are not allowed to shadow declarations, but declarations are
allowed to shadow each other, as long as there are no ambiguous
references.
closes#678
Previous commit shifted everything down in the start.zig file, and
unfortunately our stage2 test harness depends on absolute line
numbers for a couple tests.
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.
Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
Instead of referencing stub indices since these can now be obtained
in a more generic fashion from the actual linked-list of atoms in
the __stub_helper section.
This is a simple structure containing an array and a length, that can be viewed as a slice.
It is useful to pass-by-copy small data whose exact size is known at runtime, but whose maximum size is known at comptime. This greatly simplifies code that otherwise would require an allocator, or reimplementing what this type does.
The primary purpose of this change is to eliminate one usage of
`usingnamespace` in the standard library - specifically the usage for
errno values in `std.os.linux`.
This is accomplished by truncating the `E` prefix from error values, and
making errno a proper enum.
A similar strategy can be used to eliminate some other `usingnamespace`
sites in the std lib.
This is a backwards-compatible language change.
Previously, `@intToEnum` coerced its integer operand to the integer tag
type of the destination enum type, often requiring the callsite to
additionally wrap the operand in an `@intCast`. Now, the `@intCast` is
implicit, and any integer operand can be passed to `@intToEnum`.
The same as before, it is illegal behavior to pass any integer which does
not have a corresponding enum tag.
With this routine, we are now able to freely shift stub_helper
section in memory and in file since the VM addressing is now dynamically
dependent on the positioning of `__stub_helper` preamble and other
sections generated by the linker.
* Introduce `memoized_calls` to `Module` which stores all the comptime
function calls that are cached. It is keyed on the `*Fn` and the
comptime arguments, but it does not yet properly detect comptime function
pointers and avoid memoizing in this case. So it will have false
positives for when a comptime function call mutates data through a
pointer parameter.
* Sema: Add a new helper function: `resolveConstMaybeUndefVal`
* Value: add `enumToInt` method and use it in `zirEnumToInt`. It is
also used by the hashing function.
* Value: fix representation of optionals to match error unions.
Previously it would not handle nested optionals correctly. Now it
matches the memory layout of error unions and supports nested
optionals properly. This required changes in all the backends for
generating optional constants.
* TypedValue gains `eql` and `hash` methods.
* Value: Implement hashing for floats, optionals, and enums.
Additionally, the zig type tag is added to the hash, where it was not
previously, so that values of differing types will get different
hashes.
By incrementally I mean using the incremental linker machinery
and concepts. Currently, lots of repetition but already highlighted
a potential problem with resolving relocations for symbols that
weren't seen yet but wanting to write the atom to file (before
seeing the relevant atoms).
The current version of code uses isARM to check if we are compiling to any arm target then checks the target bit width to either add the 32-bit sources or 64-bit source. However, isARM only returns true for 32-bit targets, and isAARCH64 is for the 64-bit targets.
I also replaced the unreachable with a @panic when we receive an unsupported arch because this code is reachable and should turn into an error.
with relocations to `dyld_private` and `__dyld_stub_binder` symbols
making the routine properly dynamic (i.e., making it possible to call
the routine before VM allocation takes place).
clang::ASTUnit::LoadFromCommandLine interprets the first argument as
the name of program (like the main function).
This change shifts the arguments passing "" for the first argument.
The compstui library contains 4 Windows functions which were recently added to the win32metadta project. I copied this def file from the mingw-w64 project.
In some cases (such as bitcast), an operand may be the same MCValue as
the result. If that operand died and was a register, it was freed by
processDeath. We have to "re-allocate" the register.
The big change in this commit is making `semaDecl` resolve the fields if
the Decl ends up being a struct or union. It needs to do this while
the `Sema` is still in scope, because it will have the resolved AIR
instructions that the field type expressions possibly reference. We do
this after the decl is populated and set to `complete` so that a `Decl`
may reference itself.
Everything else is fixes and improvements to make the test suite pass
again after making this change.
* New AIR instruction: `ptr_elem_ptr`
- Implemented for LLVM backend
* New Type tag: `type_info` which represents `std.builtin.TypeInfo`. It
is used by AstGen for the operand type of `@Type`.
* ZIR instruction `set_float_mode` uses `coerced_ty` to avoid
superfluous `as` instruction on operand.
* ZIR instruction `Type` uses `coerced_ty` to properly handle result
location type of operand.
* Fix two instances of `enum_nonexhaustive` Value Tag not handled
properly - it should generally be handled the same as `enum_full`.
* Fix struct and union field resolution not copying Type and Value
objects into its Decl arena.
* Fix enum tag value resolution discarding the ZIR=>AIR instruction map
for the child Sema, when they still needed to be accessed.
* Fix `zirResolveInferredAlloc` use-after-free in the AIR instructions
data array.
* Fix `elemPtrArray` not respecting const/mutable attribute of pointer
in the result type.
* Fix LLVM backend crashing when `updateDeclExports` is called before
`updateDecl`/`updateFunc` (which is, according to the API, perfectly
legal for the frontend to do).
* Fix LLVM backend handling element pointer of pointer-to-array. It
needed another index in the GEP otherwise LLVM saw the wrong type.
* Fix LLVM test cases not returning 0 from main, causing test failures.
Fixes a regression introduced in
6a5094872f.
* Implement comptime shift-right.
* Implement `@Type` for integers and `@TypeInfo` for integers.
* Implement union initialization syntax.
* Implement `zirFieldType` for unions.
* Implement `elemPtrArray` for a runtime-known operand.
* Make `zirLog2IntType` support RHS of shift being `comptime_int`. In
this case it returns `comptime_int`.
The motivating test case for this commit was originally:
```zig
test "example" {
var l: List(10) = undefined;
l.array[1] = 1;
}
fn List(comptime L: usize) type {
var T = u8;
return struct {
array: [L]T,
};
}
```
However I changed it to:
```zig
test "example" {
var l: List = undefined;
l.array[1] = 1;
}
const List = blk: {
const T = [10]u8;
break :blk struct {
array: T,
};
};
```
Which ended up being a similar, smaller problem. The former test case
will require a similar solution in the implementation of comptime
function calls - checking if the result of the function call is a struct
or union, and using the child `Sema` before it is destroyed to resolve
the fields.
* Add support for recursive objects to std.json.parse
* Remove previously defined error set
* Try with function which returns an error set
* Don't analyze already inferred types
* Add comptime to inferred_type parameter
* Make ParseInternalError to accept only a single argument
* Add public `ParseError` for `parse` function
* Use error.Foo syntax for errors instead of a named error set
* Better formatting
* Update to latest code changes
The async/await documentation was somewhat hard for me to follow when first learning. Two particular sticking points were
1. The alphabet example constructing the string "abcdefghi" breaks the stated rule that every async has a matching await.
2. It was somewhat unclear to me what the rules for control flow were around async/await constructs.
I've tried to improve this documentation with some minimal explanatory edits, which are correct to the best of my beginner's understanding & experimentation.
* Make clearer inline code blocks in language reference paragraphs
This commit makes the inline code blocks within paragraphs standout against the
descriptive text. The code blocks within tables are left un-styled.
The line-height of the paragraphs has been set to 1.7 based on recommendations
from MDN Web Docs and W3C. The value is unitless based on the recommendation.
Closes#9316, #6313
* Make clearer inline code blocks in language reference paragraphs
Goal: To improve legibility of inline code blocks in the language reference.
This commit alters the styles of code HTML elements in paragraphs, unordered
lists, ordered lists, tables, and preformatted text elements.
Most of the changes here are taken from suggestions from @dbandstra on GitHub
in response to a code review.
* p, ul, ol, but not #toc are set to the same line-height
* p, ul, ol, and table have the same inline code styles
* The inline code background color set to match the preformatted code blocks in
the light theme and dark theme. The border colors are adjusted as well.
* The preformatted code block font size is set to default. The 12pt setting was removed.
The line-height of paragraphs is set to 1.5. This value is chosen based on
recommendations from W3C [1] via MDN Web Docs [2] and the contents of the
language reference.
The MDN Web Docs offers two recommendations:
1. Use a unitless number for line-height.
2. Use a minimum value of 1.5 for main paragraph content.
[1] https://www.w3.org/TR/WCAG21/#visual-presentation
[2] https://developer.mozilla.org/en-US/docs/Web/CSS/line-heightCloses#6313, #9316
* Set language reference line-height and pre code border color
This commit is an update to a group of commits with the goal of improving the
legibility of the language reference. In this commit, the line-height is now set
in the `#contents` ID and reset to normal for preformatted code blocks. This
change better separates lines of all main content text for legibility.
Closes: #9316, #6313
* Style the language reference code elements
This commit sets the style of the HTML code element. Since preformatted code
blocks have an overriding style, this is safe to set for all inline code elements.
* add Random.enumValue()
* edits suggested by review
* applied zig fmt
* Rewrite to use std.enums.values
Implemented pfgithub's suggestion to rewrite against this function, greatly simplifying the implementation.
Co-authored-by: Justin Whear <justin@economicmodeling.com>
this made errors go from stuff like:
> type comptime_int cannot represent integer value 40
to
> type u5 cannot represent integer value 40
which makes much more sense
After merging `populateMetadata` with `populateMissingMetadata`,
it is imperative we clear the number of symbols in `writeSymbolTable`.
However, this is hopefully just a temp measure until the convergence
of incremental with traditional is complete.
More precisely, aac4fe380d16a957627af2d6e5110ee35ad7e7e7 which is the
current tip of release/13.x.
Immediately following commits are tracking the same LLVM version.
- Take into account that iteration order is undefined by checking against a map instead of relying on numerically sorted iteration order
- Check both path and basename for each entry instead of just path
Allows for iterating over the kvs when constructing with a list literal instead of having to create a separate array to pass into ComptimeStringMap in order to maintain access to the values.
For example when making a set, before in order to loop over the kvs you'd have to do something like:
const MyKV = struct { @"0": []const u8 };
const kvs: []const MyKV = &[_]MyKV{ .{ @"0" = "foo"}, .{ @"0" = "bar" } };
const map = ComptimeStringMap(void, kvs);
for (kvs) |kv| {}
whereas now it's possible to do:
const map = ComptimeStringMap(void, .{ .{"foo"}, .{"bar"} });
for (map.kvs) |kv| {}
* AstGen: use coerced_ty ResultLoc on array types and rely on Sema
doing type coercion, to reduce the size of the ZIR for these
instructions.
* Sema: implement `@ptrCast`.
* Sema: implement coercion from `T` to `?T` with an intermediate
coercion rather than equality check.
The idea is to depend on this language feature as little as possible
with the hopes that it can be adjusted to be less of an anti-pattern.
This also helps self-hosted, which does not yet implement
`usingnamespace`, get closer to being able to build compiler-rt.
instead of globally storing unresolved and tentative defs,
store indices to actual symbols in the functions that are responsible
for symbol resolution.
This is now no longer limited to targeting macOS natively but also
tries to detect the sysroot when targeting different Apple platforms
from macOS; for instance targeting iPhone Simulator from macOS. In
this case, Zig will try detecting the SDK path by invoking
`xcrun --sdk iphonesimulator --show-sdk-path`, and if the command
fails because the SDK doesn't exist (case when having CLT installed only)
or not having either Xcode or CLT installed, we simply return null
signaling that the user has to provide the sysroot themselves.
For example, in order to run a binary on an iPhone Simulator,
you need to specify that explicitly as part of the target as
`aarch64-ios-simulator` rather than `aarch64-ios-gnu` or
`aarch64-ios` for short.
* ensure we correctly transfer `-iwithsysroot` and
`-iframeworkwithsysroot` flags with values from `build.zig` and that
they are correctly transferred forward to `zig cc`
* try to look for `libSystem.tbd` in the provided syslibroot - one
caveat that the user will have to specify library search paths too
Previously, I have incorrectly assumed that with two-level namespace
we only need to link in dylibs/frameworks that actually export symbols
which are undefined in the linked image. Turns out, regardless of
whether we link with two-level namespace (default on macOS) or a
flat namespace (more common on other platforms), we always need to
put the dylibs/frameworks as specified by the user from the linker
line into the final linked image.
Some parsers interpret these as octal, some don't, and the confusion can lead to vulnerabilities.
Return error.NonCanonical when parsing IPv4 addresses with 0 prefixes.
* Value: rename `error_union` to `eu_payload` and clarify the intended
usage in the doc comments. The way error unions is represented with
Value is fixed to not have ambiguous values.
* Fix codegen for error union constants in all the backends.
* Implement the AIR instructions having to do with error unions in the
LLVM backend.
* New AIR instructions: ptr_add, ptr_sub, ptr_elem_val, ptr_ptr_elem_val
- See the doc comments for details.
* Sema: implement runtime pointer arithmetic.
* Sema: implement elem_val for many-pointers.
* Sema: support coercion from `*[N:s]T` to `[*]T`.
* Type: isIndexable handles many-pointers.
* Introduce `ret_load` ZIR instruction which does return semantics
based on a corresponding `ret_ptr` instruction. If the return type of
the function has storage for the return type, it simply returns.
However if the return type of the function is by-value, it loads the
return value from the `ret_ptr` allocation and returns that.
* AstGen: improve `finishThenElseBlock` to not emit break instructions
after a return instruction in the same block.
* Sema: `ret_ptr` instruction works correctly in comptime contexts.
Same with `alloc_mut`.
The test case with a recursive inline function having an implicitly
comptime return value now has a runtime return value because of the fact
that it calls a function in a non-comptime context.
This commit solves the problem in a much simpler way: putting
runtime-known values in place of non-comptime arguments when
instantiating a generic function.
The `comptime_args` field of Fn has a clarified purpose:
For generic function instantiations, there is a `TypedValue` here
for each parameter of the function:
* Non-comptime parameters are marked with a `generic_poison` for the value.
* Non-anytype parameters are marked with a `generic_poison` for the type.
Sema now has a `fn_ret_ty` field. Doc comments reproduced here:
> When semantic analysis needs to know the return type of the function whose body
> is being analyzed, this `Type` should be used instead of going through `func`.
> This will correctly handle the case of a comptime/inline function call of a
> generic function which uses a type expression for the return type.
> The type will be `void` in the case that `func` is `null`.
Various places in Sema are modified in accordance with this guidance.
Fixed `resolveMaybeUndefVal` not returning `error.GenericPoison` when
Value Tag of `generic_poison` is encountered.
Fixed generic function memoization incorrect equality checking. The
logic now clearly deals properly with any combination of anytype and
comptime parameters.
Fixed not removing generic function instantiation from the table in case
a compile errors in the rest of `call` semantic analysis. This required
introduction of yet another adapter which I have called
`GenericRemoveAdapter`. This one is nice and simple - it's the same hash
function (the same precomputed hash is passed in) but the equality
function checks pointers rather than doing any logic.
Inline/comptime function calls coerce each argument in accordance with
the function parameter type expressions. Likewise the return type
expression is evaluated and provided (see `fn_ret_ty` above).
There's a new compile error "unable to monomorphize function". It's
pretty unhelpful and will need to get improved in the future. It happens
when a type expression in a generic function did not end up getting
resolved at a callsite. This can happen, for example, if a runtime
parameter is attempted to be used where it needed to be comptime known:
```zig
fn foo(x: anytype) [x]u8 { _ = x; }
```
In this example, even if we pass a number such as `10` for `x`, it is
not marked `comptime`, so `x` will have a runtime known value, making
the return type unable to resolve.
In the LLVM backend I implement cmp instructions for float types to pass
some behavior tests that used floats.
When doing a function call, if the return type requires comptime, the
function is analyzed as an inline/comptime call.
There is an important TODO here. I will reproduce the comment from this
commit:
> In the case of a comptime/inline function call of a generic function,
> the function return type needs to be the resolved return type based on
> the function parameter type expressions being evaluated with comptime arguments
> passed in. Otherwise, it ends up being .generic_poison and failing the
> comptime/inline function call analysis.
For now these errors are handled via `@panic` rather than `unreachable`.
These are relatively likely bugs to occur at this early stage of
development, and handling them as panics lets us ship release builds
of the compiler without worrying about undefined behavior.
Furthermore, in stage1, `@panic` is implemented to include an error
return trace, while `unreachable` is not. In this case, the error return
traces are extremely helpful in debugging the compiler.
There was a bug in stage2 regarding iteration of function parameter AST.
This resulted in a false negative "unused parameter" compile error,
which, when fixed, revealed a bug in the std lib HashMap implementation.
* ZIR encoding for function instructions have a body for the return
type. This lets Sema for generic functions do the same thing it does
for parameters, handling `error.GenericPoison` in the evaluation of
the return type by marking the function as generic.
* Sema: fix missing block around the new Decl arena finalization. This
led to a memory corruption.
* Added some floating point support to the LLVM backend but didn't get
far enough to pass any new tests.
Module has a new field `monomorphed_funcs` which stores the set of
`*Module.Fn` objects which are generic function instantiations.
The hash is based on hashes of comptime values of parameters known to be
comptime based on an explicit comptime keyword or must-be-comptime
type expressions that can be evaluated without performing monomorphization.
This allows function calls to be semantically analyzed cheaply for
generic functions which are already instantiated.
The table is updated with a single `getOrPutAdapted` in the semantic
analysis of `call` instructions, by pre-allocating the `Fn` object and
passing it to the child `Sema`.
* The `indexable_ptr_len` ZIR instruction now uses a `none_or_ref`
ResultLoc. This prevents an unnecessary `ref` instruction from being
emitted.
* Sema: Fix `analyzeCall` using the incorrect ZIR object for the
generic function callee.
* LLVM backend: `genTypedValue` supports a `Slice` type encoded with
the `decl_ref` `Value`.
AstGen result locations now have a `coerced_ty` tag which is the same as
`ty` except it assumes that Sema will do a coercion, so it does not
redundantly add an `as` instruction into the ZIR code. This results in
cleaner ZIR and about a 14% reduction of ZIR bytes.
param and param_comptime ZIR instructions now have a block body for
their type expressions. This allows Sema to skip evaluation of the
block in the case that the parameter is comptime-provided. It also
allows a new mechanism to function: when evaluating type expressions of
generic functions, if it would depend on another parameter, it returns
`error.GenericPoison` which bubbles up and then is caught by the
param/param_comptime instruction and then handled.
This allows parameters to be evaluated independently so that the type
info for functions which have comptime or anytype parameters will still
have types populated for parameters that do not depend on values of
previous parameters (because evaluation of their param blocks will return
successfully instead of `error.GenericPoison`).
It also makes iteration over the block that contains function parameters
slightly more efficient since it now only contains the param
instructions.
Finally, it fixes the case where a generic function type expression contains
a function prototype. Formerly, this situation would cause shared state
to clobber each other; now it is in a proper tree structure so that
can't happen. This fix also required adding a field to Sema
`comptime_args_fn_inst` to make sure that the `comptime_args` field
passed into Sema is applied to the correct `func` instruction.
Source location for `node_offset_asm_ret_ty` is fixed; it was pointing at
the asm output name rather than the return type as intended.
Generic function instantiation is fixed, notably with respect to
parameter type expressions that depend on previous parameters, and with
respect to types which must be always comptime-known. This involves
passing all the comptime arguments at a callsite of a generic function,
and allowing the generic function semantic analysis to coerce the values
to the proper types (since it has access to the evaluated parameter type
expressions) and then decide based on the type whether the parameter is
runtime known or not. In the case of explicitly marked `comptime`
parameters, there is a check at the semantic analysis of the `call`
instruction.
Semantic analysis of `call` instructions does type coercion on the
arguments, which is needed both for generic functions and to make up for
using `coerced_ty` result locations (mentioned above).
Tasks left in this branch:
* Implement the memoization table.
* Add test coverage.
* Improve error reporting and source locations for compile errors.
which internally calls `ArrayHashMap.pop`, however, returns `?KV`
instead and performs the bounds checking automatically.
This function correponds to `ArrayList.popOrNull` and is meant
to fill the gap for situations where we want the quick lookup offered
by the hash map with elegant ability to iterate and pop of the
container with automatic bound checking that plugs in well with
a `while`-loop such as
```zig
var map = std.ArrayHashMap(K, V).init(allocator);
map.deinit();
while (map.popOrNull()) |entry| {
// ... do something
}
assert(map.count() == 0);
```
The general strategy is that Sema will pre-map comptime arguments into
the inst_map, and then re-run the block body that contains the `param`
and `func` instructions. This re-runs all the parameter type expressions
except with comptime values populated.
In Sema, param instructions are now handled specially: they detect
whether they are comptime-elided or not. If so, they skip putting a
value in the inst_map, since it is already pre-populated. If not, then
they append to the `fields` field of `Sema` for use with the `func`
instruction.
So when the block body is re-run, a new function is generated with
all the comptime arguments elided, and the new function type has only
runtime parameters in it. TODO: give the generated Decls better names
than "foo__anon_x".
The new function is then added to the work queue to have its body
analyzed and a runtime call AIR instruction to the new function is
emitted.
When the new function gets semantically analyzed, comptime parameters are
pre-mapped to the corresponding `comptime_args` values rather than
mapped to an `arg` AIR instruction. `comptime_args` is a new field that
`Fn` has which is a `TypedValue` for each parameter. This field is non-null
for generic function instantiations only. The values are the comptime
arguments. For non-comptime parameters, a sentinel value is used. This is
because we need to know the information of which parameters are
comptime-known.
Additionally:
* AstGen: align and section expressions are evaluated in the scope that
has comptime parameters in it.
There are still some TODO items left; see the BRANCH_TODO file.
* ZIR function instructions encode the index of the block that
contains the function instruction. This allows Zig to later scan the
block and find the parameter instructions, which is needed for
semantically analyzing function bodies.
* Runtime function calls insert AIR arg instructions and then inserts
Sema inst_map entries mapping the ZIR param instructions to them.
* comptime/inline function call inserts Sema inst_map entries mapping
the ZIR param instructions to the AIR callsite arguments.
With this commit we are back to the tests passing.
#8589 introduced correct handling of signed (possibly negative) array access
of pointers. Since unadorned integer literals in C are signed, this resulted
in inefficient generated code when indexing a pointer by a non-negative
integer literal.
ZIR encoding for functions is changed in preparation for generic
function support. As an example:
```zig
const std = @import("std");
const expect = std.testing.expect;
test "example" {
var x: usize = 0;
x += checkSize(i32, 1);
x += checkSize(bool, true);
try expect(x == 5);
}
fn checkSize(comptime T: type, x: T) usize {
_ = x;
return @sizeOf(T);
}
```
Previous ZIR for the `checkSize` function:
```zir
[165] checkSize line(10) hash(0226f62e189fd0b1c5fca02cf4617562): %55 = block_inline({
%56 = decl_val("T") token_offset:11:35
%57 = as_node(@Ref.type_type, %56) node_offset:11:35
%69 = extended(func([comptime @Ref.type_type, %57], @Ref.usize_type, {
%58 = arg("T") token_offset:11:23
%59 = as_node(@Ref.type_type, %58) node_offset:11:35
%60 = arg("x") token_offset:11:32
%61 = dbg_stmt(11, 4)
```
ZIR for the `checkSize` function after this commit:
```zir
[157] checkSize line(10) hash(0226f62e189fd0b1c5fca02cf4617562): %55 = block_inline({
%56 = param_comptime("T", @Ref.type_type) token_offset:11:23
%57 = as_node(@Ref.type_type, %56) node_offset:11:35
%58 = param("x", %57) token_offset:11:32
%67 = func(@Ref.usize_type, {
%59 = dbg_stmt(11, 4)
```
Noted differences:
* Previously the type expression was redundantly repeated.
* Previously the parameter names were redundantly stored in the ZIR
extra array.
* Instead of `arg` ZIR instructions as the first instructions within a
function body, they are now outside the function body, in the same
block as the `func` instruction. There are variants:
- param
- param_comptime
- param_anytype
- param_anytype_comptime
* The param instructions additionally encode the type.
* Because of the param instructions, the `func` instruction no longer
encodes the list of parameter types or the comptime bits.
It's implied that Sema will collect the parameters so that when a `func`
instruction is encountered, they will be implicitly used to construct
the function's type. This is so that we can satisfy all 3 ways of
performing semantic analysis on a function:
1. runtime: Sema will insert AIR arg instructions for each parameter,
and insert into the Sema inst_map ZIR param => AIR arg.
2. comptime/inline: Sema will insert into the inst_map ZIR param =>
callsite arguments.
3. generic: Sema will map *only the comptime* ZIR param instructions to
the AIR instructions for the comptime arguments at the callsite, and
then re-run Sema for the function's Decl. This will produce a new
function which is the monomorphized function.
Additionally:
* AstGen: Update usage of deprecated `ensureCapacity` to
`ensureUnusedCapacity` or `ensureTotalCapacity`.
* Introduce `Type.fnInfo` for getting a bunch of data about a function
type at once, and use it in `analyzeCall`.
This commit starts a branch to implement generic functions in stage2.
Test regressions have not been addressed yet.
Remove some unused debugging machinery such as full printing of the
symtab after symbol resolution. It was there only for the time of
rewriting the linker.
This way, we can explicitly signal if a test requires the presence
of macOS SDK to build. For instance, when testing our in-house
MachO linker for correctly linking Objective-C, we require the
presence of the SDK on the host system, and we can enforce this
with `-Denable-macos-sdk` flag to `zig build test-standalone`.
On arm64 macOS, the address of the last frame is 0x0 rather than
a positive value like 0x1 on x86_64 macOS, therefore, we overflow
an integer trying to subtract 1 when printing the stack trace. This
patch fixes it by first checking for this condition before trying
to subtract 1.
Note that we do not need to signal the `SignalIterator` about this
as it will correctly detect this condition on the subsequent iteration
and return `null`, thus terminating the loop.
`func_extended` ZIR instructions now have a one of the unused flags used
as a `has_comptime_bits` boolean. When set, it means 1 or more
parameters are `comptime`. In this case, there is a u32 per every 32
parameters (usually just 1 u32) with each bit indicating whether the
corresponding parameter is `comptime`.
Sema uses this information to correctly mark generic functions as
generic. There is now a TODO compile error in place in case a generic
function call happens. A future commit will do the generic function call
implementation.
Sema now properly handles alloc_inferred and alloc_inferred_mut ZIR
instructions inside a comptime execution context. In this case it
creates Decl objects and points to them with the new `decl_ref_mut`
Value Tag. `storePtr` is updated to mutate such Decl types and values.
In this case it destroys the old arena and makes a new one, preventing
memory growth during comptime code execution.
Additionally:
* Fix `storePtr` to emit a compile error for a pointer comptime-known
to be undefined.
* Fix `storePtr` to emit runtime instructions for all the cases that a
pointer is comptime-known but does not support comptime
dereferencing, such as `@intToPtr` on a hard-coded address, or an
extern function.
* Fix `ret_coerce` not coercing inside inline function call context.
- This ensures we honor the user's integer size when performing wrapping operations.
- Also, instead of using ensureCapacity, we now use ensureUnusedCapacity.
This uses the same approach as error unions,
meaning it's a `WValue` with its tag set to `multi_value`.
The initial index of the multi_value will contain the null-tag, used to check if the value
is null or not. The other values will be the payload.
To support the `.?` shorthand syntax, we save the result from checking the null-tag
into a new local, which can then be loaded later in the block to either hit `unreachable` or
set the actual payload value.
Currently, it seems `.?` and `orelse unreachable` results in different AIR structure.
TODO: Is this expected?
- Some opcodes have the incorrect value set in std.
- Some opcodes were missing and have now been added to std.
- Adding wrapping operands for add,sub and mul.
- Implement intCast which either extends or shortens the type.
instead, immediately transfer ownership to MachO struct. Also, revert
back to try-ok-fail parsing approach of objects, archives, and dylibs.
It seems easier to try and fail than check if the file *is* of a
certain type given that a dylib may be a stub and parsing yaml
twice in a row seems very wasteful.
Hint for the future: if we optimise yaml/TAPI parsing, this approach
may be rethought!
instead, pass it in functions that require it. Also, when parsing
relocs, make Object part of the context struct where we pass in
additional goodies such as `*MachO` or `*Allocator`.
fixes a crash when lowering `a == b` and they are of type bool.
I'm not worried about floats; I think we will probably add separate AIR
instructions for floats.
When doing `x == true` or `x == false` it is now lowered as either a
no-op or a not, respectively, rather than a cmp instruction.
This commit also extracts a zirCmpEq function out from zirCmp, reducing
the amount of branching (on is_equality_cmp) in both functions.
* `Value.toType` accepts a buffer parameter instead of an allocator
parameter and can no longer fail.
* Module: remove the unused `mod: *Module` parameter from various
functions.
* `Value.compare` now accepts a `Type` parameter which indicates the
type of both operands. There is also a `Value.compareHetero` which
accepts only Value parameters and supports comparing mixed types.
Likewise, `Value.eql` requires a `Type` parameter.
* `Value.hash` is removed; instead the hash map context structs now
have a `ty: Type` field, and the hash function lives there, where it
has access to a Value's Type when it computes a hash.
- This allowed the hash function to be greatly simplified and sound
in the sense that the same Values, even with different
representations, always hash to the same thing.
* Sema: Fix source location of zirCmp when an operand is runtime known
but needs to be comptime known.
* Remove unused target parameter from `Value.floatCast`.
After this change, the frontend and backend cooperate to keep track of
which Decls are actually emitted into the machine code. When any backend
sees a `decl_ref` Value, it must mark the corresponding Decl `alive`
field to true.
This prevents unused comptime data from spilling into the output object
files. For example, if you do an `inline for` loop, previously, any
intermediate value calculations would have gone into the object file.
Now they are garbage collected immediately after the owner Decl has its
machine code generated.
In the frontend, when it is time to send a Decl to the linker, if it has
not been marked "alive" then it is deleted instead.
Additional improvements:
* Resolve type ABI layouts after successful semantic analysis of a
Decl. This is needed so that the backend has access to struct fields.
* Sema: fix incorrect logic in resolveMaybeUndefVal. It should return
"not comptime known" instead of a compile error for global variables.
* `Value.pointerDeref` now returns `null` in the case that the pointer
deref cannot happen at compile-time. This is true for global
variables, for example. Another example is if a comptime known
pointer has a hard coded address value.
* Binary arithmetic sets the requireRuntimeBlock source location to the
lhs_src or rhs_src as appropriate instead of on the operator node.
* Fix LLVM codegen for slice_elem_val which had the wrong logic for
when the operand was not a pointer.
As noted in the comment in the implementation of deleteUnusedDecl, a
future improvement will be to rework the frontend/linker interface to
remove the frontend's responsibility of calling allocateDeclIndexes.
I discovered some issues with the plan9 linker backend that are related
to this, and worked around them for now.
* AIR no longer has a `variables` array. Instead of the `varptr`
instruction, Sema emits a constant with a `decl_ref`.
* AIR no longer has a `ref` instruction. There is no longer any
instruction that takes a value and returns a pointer to it. If this
is desired, Sema must either create an anynomous Decl and return a
constant `decl_ref`, or in the case of a runtime value, emit an
`alloc` instruction, `store` the value to it, and then return the
`alloc`.
* The `ref_val` Value Tag is eliminated. `decl_ref` should be used
instead. Also added is `eu_payload_ptr` which points to the payload
of an error union, given an error union pointer.
In general, Sema should avoid calling `analyzeRef` if it can be helped.
For example in the case of field_val and elem_val, there should never be
a reason to create a temporary (alloc or decl). Recent previous commits
made progress along that front.
There is a new abstraction in Sema, which looks like this:
var anon_decl = try block.startAnonDecl();
defer anon_decl.deinit();
// here 'anon_decl.arena()` may be used
const decl = try anon_decl.finish(ty, val);
// decl is typically now used with `decl_ref`.
This pattern is used to upgrade `ref_val` usages to `decl_ref` usages.
Additional improvements:
* Sema: fix source location resolution for calling convention
expression.
* Sema: properly report "unable to resolve comptime value" for loads of
global variables. There is now a set of functions which can be
called if the callee wants to obtain the Value even if the tag is
`variable` (indicating comptime-known address but runtime-known value).
* Sema: `coerce` resolves builtin types before checking equality.
* Sema: fix `u1_type` missing from `addType`, making this type have a
slightly more efficient representation in AIR.
* LLVM backend: fix `genTypedValue` for tags `decl_ref` and `variable`
to properly do an LLVMConstBitCast.
* Remove unused parameter from `Value.toEnum`.
After this commit, some test cases are no longer passing. This is due to
the more principled approach to comptime references causing more
anonymous decls to get sent to the linker for codegen. However, in all
these cases the decls are not actually referenced by the runtime machine
code. A future commit in this branch will implement garbage collection
of decls so that unused decls do not get sent to the linker for codegen.
This will make the tests go back to passing.
- generic "struct:L:C" naming if rloc is NodeTypeStructValueField
- generic "struct:L:C" naming if rloc is NodeTypeFnCallExpr
- move some tests from test/behavior/misc to test/behavior/typename
closes#4330closes#9339
While the SysV ABI is not that complicated, LLVM does not allow us
direct access to enforce it. By mimicking the IR generated by clang,
we can trick LLVM into doing the right thing. This involves two main
additions:
1. `AGG` ABI class
This is not part of the spec, but since we have to track class per
eightbyte and not per struct, the current enum is not enough. I
considered adding multiple classes like: `INTEGER_INTEGER`,
`INTEGER_SSE`, `SSE_INTEGER`. However, all of those cases would trigger
the same code path so it's simpler to collapse into one. This class is
only used on SysV.
2. LLVM C ABI type
Clang uses different types in C ABI function signatures than the
original structs passed in, and does conversion. For example, this
struct: `{ i8, i8, float }` would use `{ i16, float }` at ABI boundaries.
When passed as an argument, it is instead split into two arguments `i16`
and `float`. Therefore, for every struct that passes ABI boundaries we
need to keep track of its corresponding ABI type. Here are some more
examples:
```
| Struct | ABI equivalent |
| { i8, i8, i8, i8 } | i32 |
| { float, float } | double |
| { float, i32, i8 } | { float, i64 } |
```
Then, we must update function calls, returns, parameter lists and inits
to properly convert back and forth as needed.
Before calling populateTestFunctions() we want to check
totalErrorCount() but that will read from some tables that might get
populated by the thread pool for C compilation tasks. So we wait until
all those tasks are finished before proceeding.
Frontend improvements:
* When compiling in `zig test` mode, put a task on the work queue to
analyze the main package root file. Normally, start code does
`_ = import("root");` to make Zig analyze the user's code, however in
the case of `zig test`, the root source file is the test runner.
Without this change, no tests are picked up.
* In the main pipeline, once semantic analysis is finished, if there
are no compile errors, populate the `test_functions` Decl with the
set of test functions picked up from semantic analysis.
* Value: add `array` and `slice` Tags.
LLVM backend improvements:
* Fix incremental updates of globals. Previously the
value of a global would not get replaced with a new value.
* Fix LLVM type of arrays. They were incorrectly sending
the ABI size as the element count.
* Remove the FuncGen parameter from genTypedValue. This function is for
generating global constants and there is no function available when
it is being called.
- The `ref_val` case is now commented out. I'd like to eliminate
`ref_val` as one of the possible Value Tags. Instead it should
always be done via `decl_ref`.
* Implement constant value generation for slices, arrays, and structs.
* Constant value generation for functions supports the `decl_ref` tag.
* Add AIR instruction: struct_field_val
- This is part of an effort to eliminate the AIR instruction `ref`.
- It's implemented for C backend and LLVM backend so far.
* Rename `resolvePossiblyUndefinedValue` to `resolveMaybeUndefVal` just
to save some columns on long lines.
* Sema: add `fieldVal` alongside `fieldPtr` (renamed from
`namedFieldPtr`). This is part of an effort to eliminate the AIR
instruction `ref`. The idea is to avoid unnecessary loads, stores,
stack usage, and IR instructions, by paying a DRY cost.
LLVM backend improvements:
* internal linkage vs exported linkage is implemented, along with
aliases. There is an issue with incremental updates due to missing
LLVM API for deleting aliases; see the relevant comment in this commit.
- `updateDeclExports` is hooked up to the LLVM backend now.
* Fix usage of `Type.tag() == .noreturn` rather than calling `isNoReturn()`.
* Properly mark global variables as mutable/constant.
* Fix llvm type generation of function pointers
* Fix codegen for calls of function pointers
* Implement llvm type generation of error unions and error sets.
* Implement AIR instructions: addwrap, subwrap, mul, mulwrap, div,
bit_and, bool_and, bit_or, bool_or, xor, struct_field_ptr,
struct_field_val, unwrap_errunion_err, add for floats, sub for
floats.
After this commit, `zig test` on a file with `test "example" {}`
correctly generates and executes a test binary. However the
`test_functions` slice is undefined and just happens to be going into
the .bss section, causing the length to be 0. The next step towards
`zig test` will be replacing the `test_functions` Decl Value with the
set of test function pointers, before it is sent to linker/codegen.
@select(
comptime T: type,
pred: std.meta.Vector(len, bool),
a: std.meta.Vector(len, T),
b: std.meta.Vector(len, T)
) std.meta.Vector(len, T)
Constructs a vector from a & b, based on the values in the predicate vector. For indices where the predicate value is true, the corresponding
element from the a vector is selected, and otherwise from b.
* properly set global variables to const if they are not a global
variable.
* implement global variable initializations.
* initial implementation of llvmType() for structs and functions.
* implement genTypedValue for variable tags
* implement more AIR instructions: varptr, slice_ptr, slice_len,
slice_elem_val, ptr_slice_elem_val, unwrap_errunion_payload,
unwrap_errunion_payload_ptr, unwrap_errunion_err,
unwrap_errunion_err_ptr.
These AIR instructions are the next blockers for `zig test` to work for
this backend.
After this commit, the "hello world" x86_64 test case passes for the
LLVM backend as well.
This is an initial version, todo:
- Also make this work for u64 values, as the table must be indexed by u32.
- Add support for signed integers.
- Add support for enums.
* There is now a main_pkg in addition to root_pkg. They are usually the
same. When using `zig test`, main_pkg is the user's source file and
root_pkg has the test runner.
* scanDecl no longer looks for test decls outside the package being
tested. honoring `--test-filter` is still TODO.
* test runner main function has a void return value rather than
`anyerror!void`
* Sema is improved to generate better AIR for for loops on slices.
* Sema: fix incorrect capacity calculation in zirBoolBr
* Sema: add compile errors for trying to use slice fields as an lvalue.
* Sema: fix type coercion for error unions
* Sema: fix analyzeVarRef generating garbage AIR
* C codegen: fix renderValue for error unions with 0 bit payload
* C codegen: implement function pointer calls
* CLI: fix usage text
Adds 4 new AIR instructions:
* slice_len, slice_ptr: to get the ptr and len fields of a slice.
* slice_elem_val, ptr_slice_elem_val: to get the element value of
a slice, and a pointer to a slice.
AstGen gains a new functionality:
* One of the unused flags of struct decls is now used to indicate
structs that are known to have non-zero size based on the AST alone.
The include_compiler_rt stored in the bin file options means that we need
compiler-rt symbols *somehow*. However, in the context of using the stage1 backend
we need to tell stage1 to include compiler-rt only if stage1 is the place that
needs to provide those symbols. Otherwise the stage2 infrastructure will take care
of it in the linker, by putting compiler_rt.o into a static archive, or linking
compiler_rt.a against an executable. In other words we only want to set this flag
for stage1 if we are using build-obj.
When using `build-exe` or `build-lib -dynamic`, `-fcompiler-rt` means building
compiler-rt into a static library and then linking it into the executable.
When using `build-lib`, `-fcompiler-rt` means building compiler-rt into an
object file and then adding it into the static archive.
Before this commit, when using `build-obj`, zig would build compiler-rt
into an object file, and then on ELF, use `lld -r` to merge it into the
main object file. Other linker backends of LLD do not support `-r` to
merge objects, so this failed with error messages for those targets.
Now, `-fcompiler-rt` when used with `build-obj` acts as if the user puts
`_ = @import("compiler_rt");` inside their root source file. The symbols
of compiler-rt go into the same compilation unit as the root source file.
This is hooked up for stage1 only for now. Once stage2 is capable of
building compiler-rt, it should be hooked up there as well.
* Added doc comments for `std.Target.ObjectFormat` enum
* `std.Target.oFileExt` is removed because it is incorrect for Plan-9
targets. Instead, use `std.Target.ObjectFormat.fileExt` and pass a
CPU architecture.
* Added `Compilation.Directory.joinZ` for when a null byte is desired.
* Improvements to `Compilation.create` logic for computing `use_llvm`
and reporting errors in contradictory flags. `-femit-llvm-ir` and
`-femit-llvm-bc` will now imply `-fLLVM`.
* Fix compilation when passing `.bc` files on the command line.
* Improvements to the stage2 LLVM backend:
- cleaned up error messages and error reporting. Properly bubble up
some errors rather than dumping to stderr; others turn into panics.
- properly call ZigLLVMCreateTargetMachine and
ZigLLVMTargetMachineEmitToFile and implement calculation of the
respective parameters (cpu features, code model, abi name, lto,
tsan, etc).
- LLVM module verification only runs in debug builds of the compiler
- use LLVMDumpModule rather than printToString because in the case
that we incorrectly pass a null pointer to LLVM it may crash during
dumping the module and having it partially printed is helpful in
this case.
- support -femit-asm, -fno-emit-bin, -femit-llvm-ir, -femit-llvm-bc
- Support LLVM backend when used with Mach-O and WASM linkers.
then, when sorting sections within segments, clear and redo the
ordinals since we re-apply them to symbols anyway. It is vital
to have the ordinals consistent with parsing and resolving relocs
however.
Some macros (for example any macro that uses token pasting) cannot be
directly translated to Zig, but may nevertheless still admit a Zig
implementation. This provides a mechanism for matching macros against
templates and mapping them to functions implemented in c_translation.zig.
A macro matches a template if it contains the same sequence of tokens, except
that the name and parameters may be renamed. No attempt is made to
semantically analyze the macro. For example the following two macros are
considered equivalent:
```C
```
But the following two are not:
```C
```
Previously, we'd filter the nlists assuming they were correctly
ordered by type: local < extern defined < undefined within the
object's symbol table but this doesn't seem to be guaranteed,
therefore, we sort by type and address in one go, and filter
defined from undefined afterwards.
The only known use case for this is the hash-to-curve operation where the top bit is always cleared.
But the function is public, so let's make it work as one would expect in the general case.
Also fix the comment by the way.
Portable Executable is an executable format, not an object format.
Everywhere in the entire zig codebase, we treated coff and pe as if they
were the same. Remove confusion by not including pe in the
std.Target.ObjectFormat enum.
For the time being, until we rewrite how atoms are handled across
linkers, store two tables in the MachO linker: one for TextBlocks
directly created and managed by the linker, and one for TextBlocks
that were spawned by Module.Decl. This allows for correct memory
clean up after linking is done.
When updating the code, I accidentally made it look at the fact that the
error set operands were a `type` rather than looking at exactly which error
set types they were.
This reverts the most recent big changes to `std.Progress` changing the
strategy for printing. Before the changes, it would leave the cursor after
the progress line, having better behavior when a stray print happened,
and supporting sub-process progress without any coordination.
After the changes, the cursor was left at the beginning of the line,
making any prints print garbage and often interfering with stack traces
or other debug information.
This commit reverts to before the changes.
Revert "std: Use more common escape sequences in Progress"
This reverts commit 8ebb18d9da.
Revert "Handle some weird edge cases of Win32 API"
This reverts commit b0724a350f.
Revert "Fix many thinkos"
This reverts commit b5a50a26eb.
Revert "Fix Progress printing on Windows systems"
This reverts commit 3010bfb08a.
Revert "std: Better handling of line-wrapping in Progress"
This reverts commit 4fc2e92876.
It incorrectly did not process the death of its operand. Additionally:
* delete dead code accidentally introduced in fe14e33945
* improve AIR printing code to include liveness data for operands.
Now an exclamation point ("!") indicates the tombstone of an AIR
instruction.
Previously we had codegen_decl for both constant values as well as
function bodies. A recent commit updated the linker backends to add
updateFunc as a separate function than updateDecl, and now this commit
does the same with work queue tasks.
The frontend now distinguishes between function pointers and function
bodies.
Thanks to this, we no longer need to do allocs per symbol name
landing in the symbol resolver, plus we do not need to actively
track if the string was already inserted into the string table.
Now supports multiple items pointing to the same body. This is a common
pattern even when using a jump table, with multiple cases pointing to
the same block of code.
In the case of a range specified, the items are moved to branches in the
else body. A future improvement may make it possible to have jump table
items as well as ranges pointing to the same block of code.
- Get correct types in wasm backend.
- `arg` is already a `Ref`, therefore simply use `@intToEnum`.
- Fix regression in `zirBoolBr, where the order of insertion was incorrect.
* Breaking language change: inline assembly must use string literal
syntax. This is in preparation for inline assembly improvements that
involve more integration with the Zig language. This means we cannot
rely on text substitution.
* Liveness: properly handle inline assembly and function calls with
more than 3 operands.
- More than 35 operands is not yet supported. This is a low priority
to implement.
- This required implementation in codegen.zig as well.
* Liveness: fix bug causing incorrect tomb bits.
* Sema: enable switch expressions that are evaluated at compile-time.
- Runtime switch instructions still need to be reworked in this
branch. There was a TODO left here (by me) with a suggestion to do
some bigger changes as part of the AIR memory reworking. Now that
time has come and I plan to honor the suggestion in a future commit
before merging this branch.
* AIR printing: fix missing ')' on alive instructions.
We're back to "hello world" working for the x86_64 backend.
- Update `fail()` to not require a `srcLoc`.
This brings it in line with other backends, and we were always passing 'node_offset = 0', anyway.
- Fix unused local due to change of architecture wrt function/decl generation.
- Replace all old instructions to indexes within the function signatures.
* some instructions are not implemented yet
* fix off-by-1 in Air.getMainBody
* Compilation: use `@import("builtin")` rather than `std.builtin`
for the values that are different for different build configurations.
* Sema: avoid calling `addType` in between
air_instructions.ensureUnusedCapacity and corresponding
appendAssumeCapacity because it can possibly add an instruction.
* Value: functions print their names
Additionally: ZIR encoding for floats now supports float literals up to
f64, not only f32. This is because we no longer need a source location
for this instruction.
Now you can pass `.unneeded` for a `LazySrcLoc` and if there ended up
being a compile error that needed it, you'll get
`error.NeededSourceLocation`.
Callsites can now exploit this error to do the expensive computation
to produce a source location object and then repeat the operation.
Now the branch is compiling again, provided that one uses
`-Dskip-non-native`, but many code paths are disabled. The code paths
can now be re-enabled one at a time and updated to conform to the new
AIR memory layout.
to the link infrastructure, instead of being stored with Module.Fn. This
moves towards a strategy to make more efficient use of memory by not
storing Air or Liveness data in the Fn struct, but computing it on
demand, immediately sending it to the backend, and then immediately
freeing it.
Backends which want to defer codegen until flush() such as SPIR-V
must move the Air/Liveness data upon `updateFunc` being called and keep
track of that data in the backend implementation itself.
Previously, this field was used because the Zir.Inst.Ref encoding
supported the concept of references to function parameters. However now
thanks to whole-file-astgen, the implementations of indexToRef and
refToIndex are trivial addition/subtraction of a comptime const integer.
It's pretty compact, with each AIR instruction only taking up 4 bits,
plus a sparse table for special instructions such as conditional branch,
switch branch, and function calls with more than 2 arguments.
This commit changes the AIR file and the documentation of the memory
layout. The actual work of modifying the surrounding code (in Sema and
codegen) is not yet done.
c_void is *not* simply `const c_void = opaque{};`. It has unique
semantics as any pointer type may coerce to `*c_void` which is not true
for an arbitrary `*opaque{}`.
Use `@` syntax to escape `_` when used as an identifier.
Remove the stage1 astgen prohibition against assigning from `_`
Note: there a few stage1 bugs preventing `_` from being used as an identifier
for a local variable or function parameter; these will be fixed by stage2.
They are unlikely to arise in real C code since identifiers starting with
underscore are reserved for the implementation.
namely, fixes proper symbol reolution when scanning and including
objects from static archives, and properly discard any null symbols
when a tentative definition was substituted by a defined, global symbol.
It makes sense to have them as a dependent type since they only ever
deal with TextBlocks. Simplify Relocations to rely on symbol indices
and symbol resolver rather than pointers.
Amends b009aca38a.
The PR predated the introduction of unused variable/constant checks,
thus the build checks weren't reporting this failure until later when
merged into master.
Also, add a solution to a degenerate case where on x86_64 a relocation
refers to a cell in a section via section start address even though
a symbol exists. In such case, make the section spawned symbol an alias
of the actual symbol.
section's alignment serving as the maximum alignment that
can be seen in this particular section. However, TextBlocks are
still allowed to have at most that alignment.
temporarily by iterating over all defined TextBlocks. However,
once we merge this with MachO incremental, updates will be done
at the point of creation and/or update.
Also, fix mining TLV knowledge for working out TLV pointers.
instead of pointer to the Symbol struct in the hope that we
can overwrite the Symbol in the object's symbol table with the
resolved Symbol later down the line.
* remove unused download page html. It's now handled in the
www.ziglang.org website repo.
* add netbsd to the downloads index.json file that we send to
the www.ziglang.org website repo.
* shallow clone the website repo to avoid downloading old copies of
data.js unnecessarily.
Implement io_uring submission queue entry preparation methods for
epoll_ctl, poll_add and poll_remove.
Poll masks are designated as 32-bit little-endian integers as
specified in liburing's definitions.
Updated io_uring_prep_rw to take in an unsigned 64-bit address instead
of an anytype. io_uring_sqe by default assumes that the address
specified in a submission queue entry is an unsigned 64-bit integer.
For stage1 ZIR instructions and stage1 AIR instructions, the instruction
op code was taking up 8 bytes due to padding even though it only needed
1 byte. This commit reduces the ref_count field from uint32_t to
uint16_t because the code only really cares if instructions are
referenced at all, not how many times they are referenced. With the
ref_count field reduced to uint16_t the uint8_t op code is now placed in
the freed up space.
Empirically, this saves 382 MiB of peak RAM usage when building the
self-hosted compiler, which is a reduction of 5%. Consequently this
resulted in a 3% reduction of cache-misses when building the self-hosted
compiler.
This was @SpexGuy's idea, committed by me because we tested it on my
computer.
* rename files to adhere to conventions
* remove unnecessary function / optionality
* fix merge conflict
* better panic message
* remove unnecessary TODO comment
* proper namespacing of declarations
* clean up documentation comments
* no copyright header needed for a brand new zig file that is not
copied from anywhere
Xoroshiro128+ is the current default non-cryptographic random
number generator.
This algorithm was designed to generate floating-point numbers, by
only using the top 53 bits. Lower bits have a significant bias, that
contradicts the documented properties for `rand.DefaultPrng`. This
also has implications on everything using `Random.fill()`, including
the way we generate random floating-point numbers.
In addition, Xoroshiro128+ has known issues. See for example:
- https://lemire.me/blog/2017/08/22/cracking-random-number-generators-xoroshiro128/
- https://www.pcg-random.org/posts/xoroshiro-fails-truncated.html
Xoshiro256++ addresses these issues, while remaining very fast.
The "Zig Test" section of the language reference has been moved between the
current "Hello World" section and the "Comments" section. This was done to
introduce the Zig test syntax before it is used in later sections.
The description of the Zig test feature has NOT been updated in this commit.
Closes#5837
* less branching by passing parameters in the main op code switch.
* properly pass the target when asking the type system for int info.
* handle u8, i16, etc when it is represented using
int_unsigned/int_signed tag.
* compile error instead of assertion failure for unimplemented cases
(greater than 64 bits integer).
* control flow cleanups
* zig.h: expand macros into inline functions
* reduce the complexity of the test case by making it one test case
that calls multiple functions. Also fix the problem of c_int max
value mismatch between host and target.
* Inferred error sets are stored in the return Type of the function,
owned by the Module.Fn. So it cleans up that memory in deinit().
* Sema: update the inferred error set in zirRetErrValue
- Update relevant code in wrapErrorUnion
* C backend: improve some some instructions to take advantage of
liveness analysis to avoid being emitted when unused.
* C backend: when an error union has a payload type with no runtime
bits, emit the error union as the same type as the error set.
AstGen had the then-else logic backwards for if expressions
on error unions. This commit fixes it.
Turns out AstGen only really needs `is_non_null` and `is_non_err`,
and does not need the `is_null` or `is_err` variants. So I removed the
`is_null{,_ptr}` and `is_err{,_ptr}` ZIR instructions (-4) and
added `is_non_err`, `is_non_err_ptr` ZIR instructions (+2) for
a total of (-2) ZIR instructions, giving us a tiny bit more headroom
within the 256 tag limit. This required swapping the order of
then/else blocks in a handful of cases, but ultimately means the
ZIR will be in the same as source order, which is convenient
when debugging.
AIR code on the other hand, gains the `is_non_err` and `is_non_err_ptr`
instructions.
Sema: fix logic in zirErrUnionCode and zirErrUnionCodePtr returning the
wrong result type.
* implement enough of ret_err_value to pass wasm tests
* only do the proper `@panic` implementation for the backends which
support it, which is currently only the C backend. The other backends
will see `@breakpoint(); unreachable;` same as before.
- I plan to do AIR memory layout reworking as a prerequisite to
fixing other backends, because that will help me put all the
constants up front, which will allow the codegen to lower to memory
without jumps.
* `@panic` is implemented using anon decls for the message. Makes it
easier on the backends. Might want to look into re-using decls for
this in the future.
* implement DWARF .debug_info for pointer-like optionals.
* ZIR: add two instructions:
- ret_err_value_code
- ret_err_value
* AstGen: add countDefers and utilize it to emit more efficient ZIR for
return expressions in the presence of defers.
* AstGen: implement |err| payloads for `errdefer` syntax.
- There is not an "unused capture" error for it yet.
* AstGen: `return error.Foo` syntax gets a hot path in return
expressions, using the new ZIR instructions. This also is part of
implementing inferred error sets, since we need to tell Sema to add
an error value to the inferred error set before it gets coerced.
* Sema: implement `@setCold`.
- Implement `@setCold` support for C backend.
* `@panic` and regular safety panics such as `unreachable` now properly
invoke `std.builtin.panic`.
* C backend: improve pointer and function value rendering.
* C linker: fix redundant typedefs.
* Add Type.error_set_inferred.
* Fix Value.format for enum_literal, enum_field_index, bytes.
* Remove the C backend test that checks for identical text
I measured a 14% reduction in Total ZIR Bytes from master branch
for std/os.zig.
Hashing, equality checking, and expanding lazy values were not
inspecting the is_comptime field of structs, causing incorrect behavior
for tuples. When looking at a comptime value of a struct, if the
is_comptime field is true, the value must be learned from the type
rather than the value.
When putting ZigValues into a hash map. The hash of a lazy value
and a fully resolved value must equal, and so we must resolve
the lazy values prior. The hash function asserts that none of
the values are lazy.
This commit intentions to have no functional changes. The only purpose
is to delete the struct IrInst, which is the common base struct that
both IrInstSrc (ZIR) and IrInstGen (AIR) instructions embed.
This untangles stage1 ZIR and AIR memory layout, paving the way for a
following commit to reduce memory usage.
It now displays the byte with proper printability handling. This makes
the relevant compile error test case no longer a regression in quality
from stage1 to stage2.
* Implement "initializing array with struct syntax"
* Implement "'_' used as an identifier without @\"_\" syntax"
* Fix source location of "missing parameter name"
* Update test cases where appropriate
In order to not regress the quality of compile errors, some improvements
had to be made.
* std.zig.parseCharLiteral is improved to return more detailed parse
failure information.
* tokenizer is improved to handle null bytes in the middle of strings,
character literals, and line comments.
* validating how many unicode escape digits in string literals is moved
to std.zig.parseStringLiteral rather than handled in the tokenizer.
* when a tokenizer error occurs, if the reported token is the 'invalid'
tag, an error note is added to point to the invalid byte location.
Further improvements would be:
- Mention the expected set of allowed bytes at this location.
- Display the invalid byte (if printable, print it, otherwise
escape-print it).
By requiring the source file to be null-terminated, we avoid extra
branching while simplifying the logic at the same time.
Running ast-check on a large zig source file (udivmodti4_test.zig),
master branch compared to this commit:
* 4% faster wall clock
* 7% fewer cache misses
* 1% fewer branches
The motivation for this commit is that there exists source files which
produce ast-check errors, but crash stage1 or otherwise trigger stage1
bugs. Previously to this commit, Zig would run AstGen, collect the
compile errors, run stage1, report stage1 compile errors and exit if
any, and then report AstGen compile errors.
The main change in this commit is to report AstGen errors prior to
invoking stage1, and in fact if any AstGen errors occur, do not invoke
stage1 at all.
This caused most of the compile error tests to fail due to things such
as unused local variables and mismatched stage1/stage2 error messages.
It was taking a long time to update the test cases one-by-one, so I
took this opportunity to unify the stage1 and stage2 testing harness,
specifically with regards to compile errors. In this way we can start
keeping track of which tests pass for 1, 2, or both.
`zig build test-compile-errors` no longer works; it is now integrated
into `zig build test-stage2`.
This is one step closer to executing compile error tests in parallel; in
fact the ThreadPool object is already in scope.
There are some cases where the stage1 compile errors were actually
better; those are left failing in this commit, to be addressed in a
follow-up commit.
Other changes in this commit:
* build.zig: improve support for -Dstage1 used with the test step.
* AstGen: minor cosmetic changes to error messages.
* stage2: add -fstage1 and -fno-stage1 flags. This now allows one to
download a binary of the zig compiler and use the llvm backend of
self-hosted. This was also needed for hooking up the test harness.
However, I realized that stage1 calls exit() and also has memory
leaks, so had to complicate the test harness by not using this flag
after all and instead invoking as a child process.
- These CLI flags will disappear once we start shipping the
self-hosted compiler as the main compiler. Until then, they can be
used to try out the work-in-progress stage2.
* stage2: select the LLVM backend by default for release modes, as long
as the target architecture is supported by LLVM.
* test harness: support setting the optimize mode
When a local variable has an initialization expression of type
'noreturn', emit a compile error. This brings this branch closer
to parity with master branch.
One more step towards lowering the memory footprint of stage1. This flag
was hiding in padding but now that it is gone we can re-arrange the
memory layout more easily.
* AstGen: implement "unreachable code" error for blocks. This works at
the statement level.
* stage1: remove the "unreachable code" error implementation, which
means removing the `is_gen` field from IrInstSrc. This is one small
step towards a smaller memory footprint for stage1. The benefits
won't be realized until a future commit because this flag took
advantage of padding.
There may be a regression here with "union has no associated enum"
error, and there is a regression with the following code:
```zig
const a = noreturn;
```
A future commit will address these regressions.
When a floating-point value with no fractional part is shoved into an
integer type we must check whether it fits or not before calling
`@floatToInt` as the builtin panics in case of overflow.
Catch the error and bubble it up to the caller.
* Add command line help for "-mexec-model"
* Define WasmExecModel enum in std.builtin.
* Drop the support for the old crt1.o in favor of crt1-command.o
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Update to accomodate the differences in Windows, which is now advisory
file locking, and include details about which operating systems have
atomic locking flags.
This logic was a workaround to prevent cache deadlocks which happened
from always using exclusive file locks. Now that the Cache system
supports sharing cached artifacts, this workaround is no longer needed.
Closes#7596
This is an extension of adding fat dylib support to zig ld, pulling out
the functionality needed to support fat headers & offsets and applying
it to zig archives.
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
When working with durations it often makes sense to use signed integers
and allow negative durations, and there is currently no nice way to
format these in std.fmt. This patch adds a simple wrapper for the
existing fmtDurtion to fit this need.
AstGen was calling findLineColumn() for every sibling Decl, using the
parent Decl as the starting point for the search for newlines. This
resulted in poor performance for large numbers of Decls with the same
parent.
The solution is simple: since AstGen progresses monotonically through
the AST, keep a single cursor into the source file, and whenever
line/column information is needed, advance the cursor. This guarantees
O(N) on the number of bytes in the file.
Perf:
As an example I ran ast-check on zigwin32/win32/everything.zig
(a 17 MiB file) in master branch, and after this commit.
With master branch, I killed the process after 17 seconds out of
boredom. With this commit, it completed in 300 milliseconds.
Closes#9234
which include:
* `__TEXT,__rodata` => `__DATA_CONST,__const`
* `__TEXT,__typelink` => `__DATA_CONST,__const`
* `__TEXT,__itablink` => `__DATA_CONST,__const`
* `__TEXT,__gosymtab` => `__DATA_CONST,__const`
* `__TEXT,__gopclntab` => `__DATA_CONST,__const`
Also, we treat section as containing machine code and mapping
it to `__TEXT,__text` if it is `S_REGULAR` and contains either
`S_ATTR_PURE_INSTRUCTIONS` or `S_ATTR_SOME_INSTRUCTIONS` or both.
With this change zig ld can link with dynamic libraries
contained within a fat/universal file that had multiple
seperate binaries embedded within it for multi-arch
support (in macOS).
Whilst zig can still only create single-architecture
executables - the ability to link with fat libraries is
useful for cases where they are the easiest (or only)
option to link against.
After giving it more thought, it doesn't make sense to separate
the two structurally. Instead, there should be two constructors
for a Dylib struct: one from binary file, and the other from a stub
file. This cleans up a lot of code and opens the way for recursive
parsing of re-exports from a dylib which are a hard requirement for
native feel when linking frameworks.
Instead of trying to fit a stub file into the frame of a Dylib struct,
I think it makes more sense to keep them as separate entities with
possibly shared interface (which would be added in the future).
This cleaned up a lot of logic in Dylib as well as Stub. Also, while
here I've made creating actual *Symbols lazy in the sense Dylib and
Stub only store hash maps of symbol names that they expose but we
defer create and referencing given dylib/stub until link time when
a symbol is actually referenced. This should reduce memory usage
and speed things up a bit.
Normally we rely on importing std to in turn import the root
in the start code, but when using the stage1 won't happen,
so in order to run AstGen on the root we put it into the
import_table here.
There is now a distinction between `@import` with a .zig extension and
without. Without a .zig extension it assumes it is a package name, and
returns error.PackageNotFound if not mapped into the package table.
This change reduces the amount of divergence in the compiler's main
pipeline logic enough to run AstGen for all files in the compilation,
regardless of whether the stage1 or stage2 backend is being used.
Practically, this means that all Zig code is subject to new compile
errors, such as unused local variables.
Additionally:
* remove leftover unsound asserts from recent hash map changes
* fix sub-Compilation errors not indenting correctly
Translate enum types as the underlying integer type. Translate enum constants
as top-level integer constants of the correct type (which does not necessarily
match the enum integer type).
If an enum constant's type cannot be translated for some reason, omit it.
See discussion https://github.com/ziglang/zig/issues/2115#issuecomment-827968279Fixes#9153
Previously the fd parameter was ignored and so the result would not get
populated. Now it passes the fd pointer to the inline assembly so that
the results can be observed.
Before this, the continue expression of a while loop did not have the
capture variable in it, making it incorrectly emit a compile error for
not using the capture, even if it was referenced.
We can just use bitcast instead of error_to_int, int_to_error since
errorToInt and intToError do not actually do anything, just change types.
This allows us to remove 2 air ops that were the exact same as bitcast
* Remove parser error on double ampersand
* Add failing test for double ampersand case
* Add error when encountering double ampersand in AstGen
"Bit and" operator should not make sense when one of its operands
is an address.
* Check that 2 ampersands are adjacent to each other in source string
* Remove cases of unused variables in tests
* Avoid emitting the copy_file_range symbol at all to prevent link-time
errors.
* Fix a bug in the check logic, the has_copy_file_range_syscall was
set to the wrong value in case of ENOSYS
* If link_libc is true don't fall-back to the raw syscall approach,
there's no policy about what to do in this case but let's follow what
the other impls do.
Fixes#9146
Previous to #7082, users could overwrite PATH_MAX in the root file to support std.os.toPosixPath, permitting the "bring your own operating system" layer to implement the POSIX API for opening files. Unfortunately that is no longer the case.
This commit intends to fix what is arguably a regression from 0.7 in a way that doesn't break any code targeting 0.8.0, making it suitable to be included in a 0.8 patch release.
However in a future release that permits breaking changes, I am of the opinion that it would be beneficial to overwrite the value, even for "supported" operating systems. Same for all the other POSIX/BYOOS functions and values. However this is beyond the scope of this commit. Further discussion of this will be made into an issue in due time.
* Don't skip the TLS initialization (Fixes#9083)
* Add a test case where a PIE program is built and run
* Refactor the common initialization code in the Linux startup
sequence.
This finishes LemonBoy's Draft PR ziglang#6750. It updates ChildProcess to collect the output from stdout/stderr asynchronously using Overlapped IO and named pipes.
`return` statements use a new function `nodeMayEvalToError` which does
some basic checks on the AST node to return never, always, or maybe.
Depending on this result, AstGen skips the errdefers, always includes
the errdefers, or emits a conditional branch to check whether the return
value is an error that Sema will have to evaluate.
Closes#8821
Unblocks #9047
Keep polling until there are enough open handles, if the child process
terminates closing the handles or explicitly closes them we just quit
polling and wait for the process handle to signal the termination
condition.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
Handle linker args joined with a = like -Wl,-rpath=foo
Update existing args --major-os-version, --minor-os-version,
--major-subsystem-version and --minor-subsytem-version to work with the
new parsing.
Also handle -Wl,--script in addition to -Wl,-T
Lakemont has no x86, no MMX, no SSE and no way of handling any fp-math. In theory LLVM is able to implicitly use the soft-float emulation library calls to legalize any such operation but, given Zig's use of many non-standard features, sometimes we hit a weak spot in the X86 codegen backend.
Consider this as a work-around for this LLVM problem, fixing the problem in LLVM is not so high in my todo list as the target is pretty niche and Intel axed it in '19.
(Commit message by @LemonBoy)
This is for consistency with the documentation on sentinel-terminated
{arrays,slices,pointers} which already use `N` for a comptime-inferred
size rather than `X`.
Also adds a behavioral test to assert that a string literal is returned.
This was already the case, but the documentation failed to point out
that the returned value is of type `*const [N:0]u8`, i.e. that of a
string literal.
Also adds a behavioral test to assert that this is the case.
NetBSD expects 2 PT_LOAD segments in a shared object, otherwise
ld.elf_so fails to load, emitting a general "not found" error.
Workaround issue by adding args `--no-rosegment` and `-znorelro`.
see #9109
This extra message was intended to help contributors by clarifying
what to do when they hit a `zig fmt` failure, but now AST errors are
also emitted here and the message may actually introduce confusion.
Remove it for now.
Previously, Zig did not properly communicate the target CPU features for
RISC-V to clang assembler, because Clang has a different way to pass CPU
features for C code and for assembly code. This commit makes Zig pass a
RISC-V -march flag in order to communicate CPU features to Clang when
compiling assembly files.
The Zig language specification will support identifiers and field access
in order to refer to which declaration to export with `@export`.
This commit implements the change in AstGen and updates the language
reference.
This new option sets a default libc paths file to be used for all
LibExeObjSteps. Setting LibExeObjStep.libc_file overrides this default.
This is required to allow users to cross compile projects linking system
libraries without needing to patch the build.zig.
This breaking change disambiguates between overriding the lib dir when
performing an installation with the Zig Build System, and overriding the
lib dir that the Zig installation itself uses.
This new option sets a default libc paths file to be used for all
LibExeObjSteps. Setting LibExeObjStep.libc_file overrides this default.
This is required to allow users to cross compile projects linking system
libraries without needing to patch the build.zig.
This breaking change disambiguates between overriding the lib dir when
performing an installation with the Zig Build System, and overriding the
lib dir that the Zig installation itself uses.
Compilation.AllError.Message interface does not provide configurable
file handle. And since all uses of this function only use stderr,
this can be added later.
This patch adjusts the exit code for a child process to be a u8. Since
the WEXITSTATUS macro returns the lower eight bits, it's safe to assume
that we can truncate the returned u32.
Don't move static local variables into the top-level scope since this
can cause name clashes if subsequently-defined variables or parameters
in different scopes share the name.
Instead, use a variable within a struct so that the variable's lexical
scope does not change. This solution was suggested by @LemonBoy
Note that a similar name-shadowing problem exists with `extern` variables
declared within block scope, but a different solution will be needed since
they do need to be moved to the top-level scope and we can't rename them.
Two bugs in the implementation ported from musl made all the complex
functions relying on ldexp return incorrect results in some cases.
Spotted in #9047
No functional changes are expected, this patch is only moving some code
in order to slim the huge bowl of spaghetti that is debug.zig.
The amount of memory leaked on error is much less than before but not
zero, some further work is required to smooth the edges of this old part
of the stdlib.
We will silently ignore expected section that are either won't take
part in linking such as any `__DWARF` section, or are known but are
not yet implemented such as `__TEXT,__eh_frame`. For any other
we will throw an error and exit.
Also, inform the caller that we currently are unable to handle
frameworks.
Up until now, we only expected old-fashioned objects which carried
two basic segments by name: __TEXT and __DATA. Since macOS 11.1,
there is a new segment __DATA_CONST, and we should expect and
correctly parse sections designated to that segment explicitly
as is the case in golang.
* fix a merge conflict discovered upon rebasing latest master
* rename Target.Cpu.Feature.Set.subSet to isSuperSetOf
* convert a comment into an assert
If there is a mismatch of CPU features provided
compared to the whitelist, then will fail the build and
print what the expected CPU model is and the feature
set for the model. Also prints what features need to be
removed.
* stage1 backend allows configuring the uwtables function attr
via a flag rather than its own logic.
* stage2 defaults to enabling uwtable attr when
linking libunwind, or always on windows
* stage2 makes link_eh_frame_hdr true automatically if uwtable
attr is set to be on for zig functions
* CLI: add -funwind-tables and -fno-unwind-tables to allow the user to
override the defaults.
* hook it up to `zig cc`
closes#9046
Before this change, when one or more of name or value are not known at
comptime, build.zig files must allocate and do the concatanation, which can be
cumbersome, and also adds a redundant allocation when name and value are
slices. The new version only does a single allocation directly in the builder's
allocator to concatonate name and value.
The origional behavior is available in defineCMacroRaw, for use in situations
such as parseing c compiler arguments.
Additionally, several places have been updated to use the new funtions.
I've added these three functions to all switches except NetBSD, because I don't know what the function is named there. I've even added it on the .windows switch since all the posix functions seem to be there anyways.
It turns out the code was not ported correctly from C and produced wrong
results for negative input values. As a bonus fix the NaN codepath by
adding yet another missing piece of code.
Spotted in #9047
The documentation (e.g. `man 7 rtnetlink`) states that ifi_change "is reserved for future use and should be always set to 0xFFFFFFFF". This is no longer true, even though the text hasn't been updated.
The only allowed system libraries that we can link are libraries
that are part of the sysroot such as libc or WASI emulated
subcomponents. This is required as Wasm allows to defer symbol
resolution until load time.
For example, the following import in Zig
```zig
extern "wasi_snapshot_preview1" fn proc_exit() void;
```
would normally result in appending `-lwasi_snapshot_preview1` flag
to the linker line. However, for Wasm/WASI, the symbol is provided
at load rather than link time, therefore, the linker should not be
concerned with resolving the symbol. As a result, we should not
consider system libs by the Wasm linker.
closes#9034
These options were listed under the
"Debug Options (Zig Compiler Development)" heading. Anything in this
section should be considered unstable and can be modified at any time
at any developer's discretion.
Zig has detection for when it is accidentally being called recursively
when trying to find the native libc installation. However it was not
working, resulting in a cryptic failure, because zig tried to execute
a command which had spaces in it rather than tokenizing it.
This improves the user experience of `zig cc` for systems that Zig
does not support cross-compiling for.
Closes#8960
I want the language reference to be divorced from any particular
community. Also remove the call to action since the docs are
known to be incomplete and are not the current focus of the project.
Closes#9055
* then, in `link/Wasm.zig` map `CRTFile` to full emulated libs name
* move logic for removing any mention of WASI snapshot
`wasi_snapshot_preview1` from `Compilation.zig` into `link/Wasm.zig`
Move parsing of system libs into `main.zig` next to where we decide
if we should link libC, and, if targeting WASI, if the specified
libname equals one of the emulated components, save it on the side
and remove it from the system libs. Then, build *only* those parts
of WASI libc that were preserved in the previous step.
This also fixes building of different crt1 bits needed to support
reactors and commands.
This replicates the expected behavior when using `clang` with
upstream `wasi-libc` sysroot: linking emulated subcomponents
such as process clocks or signals requires an explicit link flag
in the compiler invocation, for example:
```
zig cc -target wasm32-wasi -lwasi-emulated-process-clocks main.c -o main.wasm
```
This commit includes emulated libc sublibs that were not included
in the compilation and caching of WASI libc that ships with Zig.
The libs include (emulated): process clocks, getpid, mman, and signal.
With this change, it is now possible to successfully cross-compile
`wasm3` engine to WASI with `zig cc`.
For the future though, it might be worth considering splitting WASI
libc into libc-proper and modularised emulated libs as it is done
in upstream, and then have them included only if the user specifically
requests emulation/parts of it.
When linking with -lfoo syntax, this indicates to Zig that the
dependency should either be provided by Zig, or it should be dynamically
provided by the system.
For windows-gnu targets, the search path was "foo.lib". Now it
additionally looks for "libfoo.dll.a".
Closes#7799
Since v0.23 release of Wasmtime, if we want to iterate a directory
Y then directory Y needed to have been granted `fd_readdir` right.
However, it is now also required for directory X to carry `fd_readdir`
right, and so on, up-chain all the way until we reach the preopen
(which possesses all rights by default).
This caused problems for us since our libstd implementation is more
fine-grained and allowed for parent dirs not to carry the right while
allow for iterating on its children. My proposal here is to always
grant `fd_readdir` right as part of
`std.fs.Dir.OpenDirOptions.access_sub_paths`. This seems to be the
approach taken by Rust also, plus we should be justified to take this
approach since WASI is experimental and snapshot1 will be discontinued
eventually and replaced with a new approach to access management
that will require a complete rewrite of our libstd anyhow.
This small change makes working with tuple types much easier, allowing
the use of anonymous (eg. obtained with meta.ArgsTuple) tuples in more
places without the need for specifying each (quoted!) field name in the
initializer.
Add two helpers to ensure people won't ignore some edge cases such as
pointers overflowing the address space.
Also fix#8924 to some degree, the amount of unchecked alignForward is
still scary.
* std: Better handing of POLLHUP in ChildProcess
Upon hitting the EOF condition there are two main differences between
how Linux and the *BSD-derived systems behave: the former sets POLLHUP
and POLLIN and, after reading any residual data, only POLLHUP remains
set. The latter signal the EOF condition by setting both flags thus
requiring some extra checks to determine if the stream is "done".
DragonFly workaround/hack for POLLHUP is no longer required.
Closes#8969
One of the best ways you can contribute to Zig is to start using it for a
personal project. Here are some great examples:
* [Oxid](https://github.com/dbandstra/oxid) - arcade style game
* [TM35-Metronome](https://github.com/TM35-Metronome) - tools for modifying and randomizing Pokémon games
* [River](https://github.com/ifreund/river/) - a dynamic tiling wayland compositor
@@ -69,7 +68,7 @@ test and debug from a git working tree.
- `make` is typically sufficient to build zig during development iterations.
- `make install` performs a build __and__ install.
- `msbuild -p:Configuration=Release INSTALL.vcxproj` on Windows performs a
build and install. To avoid install, pass cmake option `-DZIG_SKIP_INSTALL_LIB_FILES=ON`.
build and install. To avoid install, pass cmake option `-DZIG_NO_LIB=ON`.
To test changes, do the following from the build directory:
@@ -105,9 +104,17 @@ When making changes to the compiler source code, the most helpful test step to
run is `test-behavior`. When editing documentation it is `docs`. You can find
this information and more in the `--help` menu.
#### Testing Changes to std lib
To quickly test a change to a file in the standard library, you can run zig test and specify a custom lib directory with the follow command-line argument.
```bash
./build/zig test lib/std/fmt.zig --zig-lib-dir lib --main-pkg-path lib/std
```
#### Testing Non-Native Architectures with QEMU
The Linux CI server additionally has qemu installed and sets `-Denable-qemu`.
The Linux CI server additionally has qemu installed and sets `-fqemu`.
This provides test coverage for, e.g. aarch64 even on x86_64 machines. It's
recommended for Linux users to install qemu and enable this testing option
when editing the standard library or anything related to a non-native
@@ -116,7 +123,7 @@ architecture.
##### glibc
Testing foreign architectures with dynamically linked glibc is one step trickier.
This requires enabling `-Denable-foreign-glibc=/path/to/glibc/multi/install/glibcs`.
This requires enabling `--glibc-runtimes /path/to/glibc/multi/install/glibcs`.
This path is obtained by building glibc for multiple architectures. This
process for me took an entire day to complete and takes up 65 GiB on my hard
drive. The CI server does not provide this test coverage. Instructions for
@@ -128,7 +135,7 @@ It's understood that most contributors will not have these tests enabled.
#### Testing Windows from a Linux Machine with Wine
When developing on Linux, another option is available to you: `-Denable-wine`.
When developing on Linux, another option is available to you: `-fwine`.
This will enable running behavior tests and std lib tests with Wine. It's
recommended for Linux users to install Wine and enable this testing option
when editing the standard library or anything Windows-related.
@@ -136,7 +143,7 @@ when editing the standard library or anything Windows-related.
#### Testing WebAssembly using wasmtime
If you have [wasmtime](https://wasmtime.dev/) installed, take advantage of the
`-Denable-wasmtime` flag which will enable running WASI behavior tests and std
`-fwasmtime` flag which will enable running WASI behavior tests and std
lib tests. It's recommended for all users to install wasmtime and enable this
testing option when editing the standard library and especially anything
message(FATAL_ERROR"LLVM (according to ${LLVM_CONFIG_EXE}) is missing target ${TARGET_NAME}. Zig requires LLVM to be built with all default targets enabled.")
endif()
endfunction(NEED_TARGET)
NEED_TARGET("AArch64")
NEED_TARGET("AMDGPU")
NEED_TARGET("ARM")
NEED_TARGET("AVR")
NEED_TARGET("BPF")
NEED_TARGET("Hexagon")
NEED_TARGET("Lanai")
NEED_TARGET("Mips")
NEED_TARGET("MSP430")
NEED_TARGET("NVPTX")
NEED_TARGET("PowerPC")
NEED_TARGET("RISCV")
NEED_TARGET("Sparc")
NEED_TARGET("SystemZ")
NEED_TARGET("WebAssembly")
NEED_TARGET("X86")
NEED_TARGET("XCore")
# Save the error message, in case this is the last llvm-config we find
list(APPENDLLVM_CONFIG_ERROR_MESSAGES"LLVM (according to ${LLVM_CONFIG_EXE}) is missing target ${TARGET_NAME}. Zig requires LLVM to be built with all default targets enabled.")
/// "These functions return a value less than or equal to zero if neither argument is NaN,
/// and a is less than or equal to b."
pubfn__lehf2(a:f16,b:f16)callconv(.C)i32{
return__cmphf2(a,b);
}
/// "These functions return zero if neither argument is NaN, and a and b are equal."
/// Note that due to some kind of historical accident, __eqhf2 and __nehf2 are defined
/// to have the same return value.
pubfn__eqhf2(a:f16,b:f16)callconv(.C)i32{
return__cmphf2(a,b);
}
/// "These functions return a nonzero value if either argument is NaN, or if a and b are unequal."
/// Note that due to some kind of historical accident, __eqhf2 and __nehf2 are defined
/// to have the same return value.
pubfn__nehf2(a:f16,b:f16)callconv(.C)i32{
return__cmphf2(a,b);
}
/// "These functions return a value less than zero if neither argument is NaN, and a
/// is strictly less than b."
pubfn__lthf2(a:f16,b:f16)callconv(.C)i32{
return__cmphf2(a,b);
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
