The fstat,lstat,stat,mknod stubs used to build older (before v2.33)
glibc versions depend on the weak_hidden_alias macro. It was removed
from the glibc libc-symbols header, so patch it back in for the older
builds.
The scope of libc_nonshared.a was greatly changed in glibc 2.33 and
2.34, but only the change from 2.34 was reflected so far. Glibc 2.33
finally switched to versioned symbols for stat functions, meaning that
libc_nonshared.a no longer contains them since 2.33. Relevant files were
therefore reverted to 2.32 versions and renamed accordingly.
This commit also removes errno.c, which was probably added to
libc_nonshared.a based on a wrong assumption that glibc/include/errno.h
requires glibc/csu/errno.c. In reality errno.h should refer to
__libc_errno (not to be confused with the public __errno_location),
which should be imported from libc.so. The inclusion of errno.c resulted
in wrong compile options as well; this commit fixes them as well.
These are tripping on 32-bit x86 but are intended to prevent glibc
itself from being built with a bad configuration. Zig is only using this
file to create libc_nonshared.a, so it's not relevant.
This is the only place in all of glibc that this macro is referenced.
What is it doing? Only preventing fstatat.c from knowing the type
definition of `__time64_t`, apparently.
Fixes compilation of fstatat.c on 32-bit x86.
I could have just included the file from upstream glibc, but it was too
silly so I just inlined it. This patch could be dropped in a future
glibc update if desired. If omitted it will cause easily solvable
C compilation failures building glibc nonshared.
- `fcntl` was renamed to `fcntl64` in glibc 2.28 (see #9485)
- `res_{,n}{search,query,querydomain}` became "their own" symbols since
glibc 2.34: they were prefixed with `__` before.
This PR makes it possible to use `fcntl` with glibc 2.27 or older and
the `res_*` functions with glibc 2.33 or older.
These patches will become redundant with universal-headers and can be
dropped. But we have to do with what we have now.
- `fcntl` was renamed to `fcntl64` in glibc 2.28 (see #9485)
- `res_{,n}{search,query,querydomain}` became "their own" symbols since
glibc 2.34: they were prefixed with `__` before.
This PR makes it possible to use `fcntl` with glibc 2.27 or older and
the `res_*` functions with glibc 2.33 or older.
These patches will become redundant with universal-headers and can be
dropped. But we have to do with what we have now.
This is a patch to glibc features.h which makes
_DYNAMIC_STACK_SIZE_SOURCE undefined unless the version is >= 2.34.
This feature was introduced with glibc 2.34 and without this patch, code
built against these headers but then run on an older glibc will end up
making a call to sysconf() that returns -1 for the value of SIGSTKSZ
and MINSIGSTKSZ.
The actual `zig objcopy` does not accept keeping multiple sections.
If you pass multiple `-j .section` arguments to `zig objcopy`, it will
only respect the last one passed.
Originally I changed `zig objcopy` to accept multiple sections
and then concatenate them instead of returning after outputting the
first section (see emitElf) but I realized concatenating probably doesn't make sense.
Fixes a regression introduced in 67455c5e70. The `errdefer` cannot run since its not possible for an error to occur, and we don't want it to run on the last handle, so we move the closing back down to where it was before 67455c5e70.
Most of the functions related to points on the Edwards25519 curve
check that input points are not in a small-order subgroup.
They don't check that points are on the prime-order subgroup,
because this is expensive, and not always necessary.
However, applications may require such a check in order to
ensure that a public key is valid, and that a secret key counterpart
exists.
Many functions in the public API of libsodium related to arithmetic
over Edwards25519 also do that check unconditionally. This is
expensive, but a good way to catch bugs in protocols and
implementations.
So, add a `rejectUnexpectedSubgroup()` function to achieve this.
The documentation on the edwards25519->curve25519 conversion
function was also updated, in order to explain how to match
libsodium's behavior if necessary.
We use an addition chain to multiply the point by the order of
the prime group.
An alternative we may implement later is Pornin's point halving
technique: https://eprint.iacr.org/2022/1164.pdf
On Windows, the console mode flag `ENABLE_VIRTUAL_TERMINAL_PROCESSING` determines whether or not ANSI escape codes are parsed/acted on. On the newer Windows Terminal, this flag is set by default, but on the older Windows Console, it is not set by default, but *can* be enabled (since Windows 10 RS1 from June 2016).
The new `File.getOrEnableAnsiEscapeSupport` function will get the current status of ANSI escape code support, but will also attempt to enable `ENABLE_VIRTUAL_TERMINAL_PROCESSING` on Windows if necessary which will provide better/more consistent results for things like `std.Progress` and `std.io.tty`.
This type of change was not done previously due to a mistaken assumption (on my part) that the console mode would persist after the run of a program. However, it turns out that the console mode is always reset to the default for each program run in a console session.
* Newer versions of Windows added VT seq support not only in Windows Terminal, but also in the old-fashioned Windows Console (standalone conhost.exe), though not enabled by default.
* Try setting the newer console mode flags provides better experience for Windows Console users.
Co-authored-by: Kexy Biscuit <kexybiscuit@biscuitt.in>
Instead of introducing YES_COLOR, a completely new standard, into the mix
it might make more sense to instead tag along with the CLICOLOR_FORCE env var,
which dates back to at least 2000 with FreeBSD 4.1.1 and which is
supported by tools like CMake.
<https://bixense.com/clicolors/>
The \r\n is necessary to get the progress tree to work properly in the old console when ENABLE_VIRTUAL_TERMINAL_PROCESSING and DISABLE_NEWLINE_AUTO_RETURN are set.
The line_upper_bound_len fix addresses part of #20161
This changes the terminal display to keep the cursor at the top left of
the progress display, so that unlocked stderr writes, perhaps by child
processes, don't get eaten by the clear.
The docs for setting stdio to "inherit" say:
It also means that this step will obtain a global lock to prevent other
steps from running in the meantime.
The implementation of this lock was missing but is now provided by this
commit.
closes#20119
Reduce node_storage_buffer_len from 200 to 83. This makes messages over
the pipe fit in a single packet (4096 bytes). There is now a comptime
assert to ensure this. In practice this is plenty of storage because
typical terminal heights are significantly less than 83 rows.
Handling of split reads is fixed; instead of using a global
`remaining_read_trash_bytes`, the value is stored in the "saved
metadata" for the IPC node.
Saved metadata is split into two arrays so that the "find" operation can
quickly scan over fds for a match, looking at 332 bytes maximum, and
only reading the memory for the other data upon match. More typical
number of bytes read for this operation would be 0 (no child processes),
4 (1 child process), or 64 (16 child processes reporting progress).
Removed an align(4) that was leftover from an older design.
This also includes part of Jacob Young's not-yet-landed patch that
implements `writevNonblock`.
rather than ignoring specifically "zig-cache" and "zig-out". The latter
is not necessarily the install prefix and should not be special.
The former will be handled by renaming zig-cache to .zig-cache.
3a3d2187f9 unintentionally broke some of the Windows console API implementation.
- The 'marker' character was no longer being written at all
- The ANSI escape codes for syncing were being written unconditionally
Clarify the usage of .paths in build.zig.zon. Follow the recommendation
of the comments to explicitly list paths by explicitly listing the paths
in the init project.
* Merge a bunch of related state together into TerminalMode. Windows
sometimes follows the same path as posix via ansi_escape_codes,
sometimes not.
* Use a different thread entry point for Windows API but share the same
entry point on Windows when the terminal is in ansi_escape_codes mode.
* Only clear the terminal when the stderr lock is held.
* Don't try to clear the terminal when nothing has been written yet.
* Don't try to clear the terminal in IPC mode.
* Fix size detection logic bug under error conditions.
7281cc1d839da6e84bb76fadb2c1eafc22a82df7 did not solve the problem
because even when Node.index is none, it still counts as initializing
the global Progress object. Just use a normal zig optional, and all is
good.
The update thread was sometimes reading the special state and then
incorrectly getting 0 for the file descriptor, making it hang since it
tried to read from stdin.
Slightly slower refresh rate. It's still updating very quickly.
Lower the initial delay so that CLI applications feel more responsive.
Even though the application is doing work for the full 500ms until
something is displayed on the screen, it feels nicer to get the progress
earlier.
Introduces `disable_zig_progress` which prevents the build runner from
assigning the child process a progress node.
This is needed for the empty_env test which requires the environment to
be completely empty.
This makes it so that any other threads which are writing to stderr have
a chance to finish before the process terminates. It also clears the
terminal in case any progress has been written to stderr, while still
accomplishing the goal of not waiting until the update thread exits.
when the root progress node has a zero length name, the sub-tree is
flattened one layer, reducing visual noise, as well as bytes written to
the terminal.
Split newline_count into written_newline_count and
accumulated_newline_count. This handle the case when the tryLock() fails
to obtain the lock, because in such case there would not be any newlines
written to the terminal but the system would incorrectly think there
were. Now, written_newline_count is only adjusted when the write() call
succeeds.
Furthermore, write() call failure is handled by exiting the update
thread.
Don't truncate trailing newline. This better handles stray writes to
stderr that are not std.Progress-aware, such as from non-zig child
processes.
This commit also makes `Node.start` and `Node.end` bail out early with a
comptime branch when it is known the target will not be spawning an
update thread.
Switch Node.Parent, Node.Index, and Node.OptionalIndex to be backed by
u8 rather than u16. This works fine since we use 200 as the preallocated
node buffer. This has the nice property that scanning the entire parents
array for allocated nodes fits in 4 cache lines, even if we bumped the
200 up to 254 (leaving room for the two special states).
The thread that reads progress updates from the pipe now handles short
reads by ignoring messages that are sent in multiple reads.
When checking the terminal size, if there is a failure, fall back to a
conservative guess of 80x25 rather than panicking. A debug message is
also emitted which would be displayed only in a debug build.
This accomplishes 2 things simultaneously:
1. Don't trust child process data; if the data is outside the expected
range, ignore the data.
2. If there is too much data to fit in the preallocated buffers, drop
the data.
Instead of making static buffers configurable, let's pick strong
defaults and then use the update thread's stack memory to store the
preallocations. The thread uses a fairly shallow stack so this memory is
otherwise unused. This also makes the data section of the executable
smaller since it runtime allocates the memory when a `std.Progress`
instance is allocated, and in the case that the process is not connected
to a terminal, it never allocates the memory.
* correctly report time spent analyzing function bodies
* print fully qualified decl names
* also have a progress node for codegen
The downside of these changes is that it's a bit flickerey, but the
upside is that it's accurate; you can see what the compiler's doing!
This fix is already in master branch for stdin, stdout, and stderr; this
commit solves the same problem but for the progress pipe.
Both fixes were originally included in one commit on this branch,
however it was split it into two so that master branch could receive the
fix before the progress branch is merged.
It stored some metadata into the canonical node storage data but that is
a race condition because another thread recycles those nodes.
Also, keep the parent name for empty child root node names.
* bump default statically allocated resources
* debug help when multiple instances of std.Progress are initialized
* only handle sigwinch on supported operating systems
* handle when reading from the pipe returns 0 bytes
* avoid printing more lines than rows
This time, we preallocate a fixed set of nodes and have the user-visible
Node only be an index into them. This allows for lock-free management of
the node storage.
Only the parent indexes are stored, and the update thread makes a
serialized copy of the state before trying to compute children lists.
The update thread then walks the tree and outputs an entire tree of
progress rather than only one line.
There is a problem with clearing from the cursor to the end of the
screen when the cursor is at the bottom of the terminal.
New design ideas:
* One global instance, don't try to play nicely with other instances
except via IPC.
* One process owns the terminal and the other processes communicate via
IPC.
* Clear the whole terminal and use multiple lines.
What's implemented so far:
* Query the terminal for size.
* Register a SIGWINCH handler.
* Use a thread for redraws.
To be done:
* IPC
* Handling single threaded targets
* Porting to Windows
* More intelligent display of the progress tree rather than only using
one line.
You don't know if it's possible to run a binary until you try. The build
system already integrates with executors and has the
`skip_foreign_checks` for exactly this use case.
- Used `Self` instead of `*const Self` where appropriate (orignally proposed in #19770)
- Replaced `@intFromPtr` and `@ptrFromInt` with `@ptrCast`, `@alignCast`, and pointer arithmetic where appropriate
With this, the only remaining instance on pointer-int conversion in hash_map.zig is in `HashMapUnmanaged.removeByPtr`, which easily be able to be eliminated once pointer subtraction is supported.
The added comment explains the issue here relatively well. The new
progress API made this bug obvious because it became visibly clear that
certain Compile steps were seemingly "hanging" until other steps
completed. As it turned out, these child processes had raced to spawn,
and hence one had inherited the other's stdio pipes, meaning the `poll`
call in `std.Build.Step.evalZigProcess` was not identifying the child
stdout as closed until an unrelated process terminated.
This reverts commit a7de02e052.
This did not implement the accepted proposal, and I did not sign off
on the changes. I would like a chance to review this, please.
Fixes regression introduced by 5d5e89aa8d
Turns out since landing that PR we haven't run any tests requiring
symlinks or any Apple SDK on a macOS host. Not great.
ArrayList uses `items` slice to store len initialized items, while
PriorityQueue stores `capacity` potentially uninitialized items.
This is a surprising difference in the API that leads to bugs!
https://github.com/tigerbeetle/tigerbeetle/pull/1948
The wrong `size_class` was used when fetching stack traces from empty
buckets. The `size_class` would always be the maximum value after
exhausting the search of active buckets rather than the actual
`size_class` of the allocation.
Empty buckets have their `alloc_cursor` set to `slot_count` to allow the
size class to be calculated later. This happens deep within the free
function.
This adds a helper and a test to verify that the size class of empty
buckets is indeed recoverable.
Not really useful after old C++ compiler removal, and
zig build has own cache system. If someone still wants it,
`CMAKE_C_COMPILER_LAUNCHER` and `CMAKE_CXX_COMPILER_LAUNCHER` exist.
From CMake docs:
> RULE_LAUNCH_COMPILE
> Note: This property is intended for internal use by ctest(1).
> Projects and developers should use the <LANG>_COMPILER_LAUNCHER
> target properties or the associated CMAKE_<LANG>_COMPILER_LAUNCHER
> variables instead.
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
Set `ZIG_PIE` default to be same as `CMAKE_POSITION_INDEPENDENT_CODE`, and
add check for situation when `ZIG_PIE` is set to True but CMake does not
support compiling position independent code. CMake's support is needed
for "zigcpp" target.
Also remove temporary variables for constructing `ZIG_BUILD_ARGS`,
instead use `list(APPEND ...)` functions.
Also remove long unused `ZIG_NO_LANGREF` variable.
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
Replace `CMAKE_SOURCE_DIR` and `CMAKE_BUILD_DIR` with different variables,
or in some cases just remove them.
For some function arguments, prepended `CMAKE_SOURCE_DIR` was removed without
replacement. This includes:
* Sources for `add_library` and `add_executable` (`ZIG_CPP_SOURCES` and `ZIG_WASM2C_SOURCES`)
* Inputs for `configure_file` and `target_include_directory`
* For arguments above, CMake already prepends
`CMAKE_CURRENT_SOURCE_DIR` to them by default, if they are relative paths.
Additionaly, it was removed from arguments of commands that have `WORKING_DIRECTORY` set to
`PROJECT_SOURCE_DIR`, they will be similarly converted by CMake for us.
Also:
* Move project declaration to the top so that these variables are
available earlier.
* Avoid calling "git" executable if ".git" directory does not exist.
* Swap "--prefix" and `ZIG_BUILD_ARGS` arguments in cmake/install.cmake
to match same "zig2 build" command in CMakeLists.txt and allow
overriding "--prefix" argument
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
* Localize most of the global properties and functions, for some time
they are only needed for "zigcpp" static library (sometimes with PUBLIC
keyword, so that it will propagate to zig2): `CMAKE_*_OUTPUT_DIRECTORY`
and two calls to `include_directories`. This removes useless flags when
building other targets and cleans build log a bit.
* Remove `EXE_CXX_FLAGS` variable, instead use more appropriate specific
properties and functions for this target. This gives better errors if
compiler does not support some of them, and CMake also handles for us
duplicate flags. It's also easier to read side-by-side with same
flags from build.zig .
* Add some comments.
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
Previously the error had a note suggesting to use `try`, `catch`, or
`if`, even for error sets where none of those work.
Instead, in case of an error set the way you can handle the error
depends very much on the specific case. For example you might be in a
`catch` where you are discarding or ignoring the error set capture
value, in which case one way to handle the error might be to `return`
the error.
So, in that case, we do not attach that error note.
Additionally, this makes the error tell you what kind of an error it is:
is it an error set or an error union? This distinction is very relevant
in how to handle the error.
Maybe I'm just being pedantic here (most likely) but I don't like how we're
just telling the user here how to "suppress this error" by "assigning the value to '_'".
I think it's better if we use the word "discard" here which I think is
the official terminology and also tells the user what it actually means
to "assign the value to '_'".
Also, using the value would also be a way to "suppress the error".
It is just one of the two options: discard or use.
* Revert "Revert "Merge pull request #19349 from nolanderc/save-commit""
This reverts commit 6ca4ed5948.
* update to new URI changes, rework `--save` type
* initialize `latest_commit` to null everywhere
this one is even harder to document then the last large overhaul.
TLDR;
- split apart Emit.zig into an Emit.zig and a Lower.zig
- created seperate files for the encoding, and now adding a new instruction
is as simple as just adding it to a couple of switch statements and providing the encoding.
- relocs are handled in a more sane maner, and we have a clear defining boundary between
lea_symbol and load_symbol now.
- a lot of different abstractions for things like the stack, memory, registers, and others.
- we're using x86_64's FrameIndex now, which simplifies a lot of the tougher design process.
- a lot more that I don't have the energy to document. at this point, just read the commit itself :p
this commit is a little too large to document fully, however the main gist of it this
- finish the `genInlineMemcpy` implement
- rename `setValue` to `genCopy` as I agree with jacob that it's a better name
- add in `genVarDbgInfo` for a better gdb experience
- follow the x86_64's method for genCall, as the procedure is very similar for us
- add `airSliceLen` as it's trivial
- change up the `airAddWithOverflow implementation a bit
- make sure to not spill of the elem_ty is 0 size
- correctly follow the RISC-V calling convention and spill the used calle saved registers in the prologue
and restore them in the epilogue
- add `address`, `deref`, and `offset` helper functions for MCValue. I must say I love these,
they make the code very readable and super verbose :)
- fix a `register_manager.zig` issue where when using the last register in the set, the value would overflow at comptime.
was happening because we were adding to `max_id` before subtracting from it.
the truncation panic logic is generated in Sema, so I don't need to roll anything
of my own. I add all of the boilerplate for that detecting the truncation and it works
in basic test cases!
this provides a much better indication of when we are having a controlled panic with an error message
or when we are actually segfaulting, as before the `trap` as causing a segfault.
when the struct is in stack memory, we access it using a byte-offset,
because that's how the stack works. on the other hand when the struct
is in a register, we are working with bits and the field offset should
be a bit offset.
- Added the basic framework for panicing with an overflow in `airAddWithOverflow`, but there is no check done yet.
- added the `cmp_lt`, `cmp_gte`, and `cmp_imm_eq` MIR instructions, and their respective functionality.
lots of thinking later, ive begun to grasp my head around how the pointers should work. this commit allows basic pointer loading and storing to happen.
- before we were storing each arg in it's own function arg register. with this commit now we store the args in the fa register before calling as per the RISC-V calling convention, however as soon as we enter the callee, aka in airArg, we spill the argument to the stack. this allows us to spend less effort worrying about whether we're going to clobber the function arguments when another function is called inside of the callee.
- we were actually clobbering the fa regs inside of resolveCallingConvetion, because of the null argument to allocReg. now each lock is stored in an array which is then iterated over and unlocked, which actually aids in the first point of this commit.
this was an annoying one to do, as there is no (to my knowledge) myriad sequence
that will allow us to do `gte` compares with an immediate without allocating a register.
RISC-V provides a single instruction to do compares, that being `lt`, and so you need to
use more than one for other variants, but in this case, i believe you need to allocate a register.
- implement `airArrayElemVal` for arrays on the stack. This is really easy
as we can just move the offset by the bytes into the array. This only works
when the index access is comptime-known though, this won't work for runtime access.
the current implementation only works when the struct is in a register. we use some shifting magic
to get the field into the LSB, and from there, given the type provenance, the generated code should
never reach into the bits beyond the bit size of the type and interact with the rest of the struct.
we use a code offset map in Emit.zig to pre-compute what byte offset each MIR instruction is at. this is important because they can be
of different size
- rename setRegOrMem -> setValue
- a naive method of passing arguments by register
- gather the prologue and epilogue and generate them in Emit.zig. this is cleaner because we have the final stack size in the emit step.
- define the "fa" register set, which contains the RISC-V calling convention defined function argument registers
Information about installed MSVC instances are stored in `state.json` files within a `Packages/_Instances` directory. The default location for this is `%PROGRAMDATA%\Microsoft\VisualStudio\Packages\_Instances`. However, it is possible for the Packages directory to be put somewhere else. In that case, the registry value `HKLM\SOFTWARE\Microsoft\VisualStudio\Setup\CachePath` is set and contains the path to the Packages directory.
Previously, WindowsSdk did not check that registry value. After this commit, the registry value `HKLM\SOFTWARE\Microsoft\VisualStudio\Setup\CachePath` is checked first, which matches what ISetupEnumInstances does (according to a Procmon log).
Clang 17 passed struct{f128} parameters using rdi and rax, while Clang
18 matches GCC 13.2 behavior, passing them using xmm0.
This commit makes Zig's LLVM backend match Clang 18 and GCC 13.2. The
commit deletes a hack in x86_64/abi.zig which miscategorized f128 as
"memory" which obviously disagreed with the spec.
LLVMABIAlignmentOfType(i128) reports 16 on this target, however the C
ABI uses align(4).
Clang in LLVM 17 does this:
%struct.foo = type { i32, i128 }
Clang in LLVM 18 does this:
%struct.foo = type <{ i32, i128 }>
Clang is working around the 16-byte alignment to use align(4) for the C
ABI by making the LLVM struct packed.
release/18.x branch, commit 78b99c73ee4b96fe9ce0e294d4632326afb2db42
This adds the flag `-D_LIBCPP_HARDENING_MODE` which is determined based
on the Zig optimization mode.
This commit also fixes libunwind, libcxx, and libcxxabi to properly
report sub compilation errors.
LLVM now refuses to lower arguments and return values on x86 targets
when the total vector bit size is >= 512.
This code detects such a situation and uses byref instead of byval.
* some manual fixes to generated CPU features code. In the future it
would be nice to make the script do those automatically.
* add to various target OS switches. Some of the values I was unsure of
and added TODO panics, for example in the case of spirv CPU arch.
New OSs:
* XROS
* Serenity
* Vulkan
Removed OSs:
* Ananas
* CloudABI
* Minix
* Contiki
New CPUs:
* spirv
The removed stuff is removed from LLVM but not Zig.
This allows running commands that take an output directory argument. The
main thing that was needed for this feature was generated file subpaths,
to allow access to the files in a generated directory. Additionally, a
minor change was required to so that the correct directory is created
for output directory args.
* `doc/langref` formatting
* upgrade `.{ .path = "..." }` to `b.path("...")`
* avoid using arguments named `self`
* make `Build.Step.Id` usage more consistent
* add `Build.pathResolve`
* use `pathJoin` and `pathResolve` everywhere
* make sure `Build.LazyPath.getPath2` returns an absolute path
This replaces `extra_file_dependencies` with support for lazy paths.
Also assert output file basenames are not empty, avoid improper use of
field default values, ensure stored strings are duplicated, and
prefer `ArrayListUnmanaged` to discourage misuse of direct field access
which wouldn't add step dependencies.
This was a "fake" type used to handle C varargs parameters, much like
generic poison. In fact, it is treated identically to generic poison in
all cases other than one (the final coercion of a call argument), which
is trivially special-cased. Thus, it makes sense to remove this special
tag and instead use `generic_poison_type` in its place. This fixes
several bugs in Sema related to missing handling of this tag.
Resolves: #19781
This function accepts a WaitGroup parameter and manages the reference
counting therein. It also is infallible.
The existing `spawn` function is still handy when the job wants to
further schedule more tasks.
closes#19803 by changing quota from (30 * N) to (10 * N * log2(N)) where
N = kvs_list.len
* adds reported adversarial test case
* update doc comment of getLongestPrefix()
Adds an `include_paths` field to RcSourceFile that takes a slice of LazyPaths. The paths are resolved and subsequently appended to the -rcflags as `/I <resolved path>`.
This fixes an accidental regression from https://github.com/ziglang/zig/pull/19174. Before that PR, all Win32 resource compilation would inherit the CC flags (via `addCCArgs`), which included things like include directories. After that PR, though, that is no longer the case.
However, this commit intentionally does not restore the previous behavior (inheriting the C include paths). Instead, each .rc file will need to have its include paths specified directly and the include paths only apply to one particular resource script. This allows more fine-grained control and has less potentially surprising behavior (at the cost of some convenience).
Closes#19605
This function incorrectly assumed that module name subsections, function
name subsections, and local name subsections are encoded the same,
however according to
[the specification](https://webassembly.github.io/spec/core/appendix/custom.html)
they are encoded differently.
This commit adds support for parsing module name subsections correctly,
which started appearing after upgrading to LLVM 18.
As of Clang 18, calling memcpy() with a misaligned pointer trips UBSAN,
even if the length is zero. This unfortunately includes any call to
`@memcpy` when source or destination are undefined and the length is
zero.
This patch makes the C backend avoid calling memcpy when the length is
zero, thereby avoiding undefined behavior.
A zig1.wasm update will be needed in the llvm18 branch to activate this
code.
wheras on NetBSD, only 2 PT_LOAD are usually produced by other compilers
(tested with host gcc and clang).
$ ldd -v main_4segs
.../main_4segs: wrong number of segments (4 != 2)
.../main_4segs: invalid ELF class 2; expected 1
* std.crypto.hash.sha2: generalize sha512 truncation
Replace `Sha512224`, `Sha512256`, and `Sha512T224` with
`fn Sha512Truncated(digest_bits: comptime_int)`.
This required refactoring `Sha2x64(comptime params)` to
`Sha2x64(comptime iv: [8]u64, digest_bits: comptime_int)`
for user-specified `digest_bits`.
I left #19697 alone but added a compile-time check that digest_bits is
divisible by 8.
Remove docs which restate type name. Add module docs and reference where
IVs come from.
* std.crypto.sha2: make Sha512_224 and Sha512_256 pub
* make generic type implementation detail, add comments
* fix iv
* address @jedisct1 feedback
* fix typo
* renaming
* add truncation clarifying comment and Sha259T192 tests
* Fix the ELF binaries for freestanding target created with the self-hosted linker.
The ELF specification (generic ABI) states that ``loadable process segments must have congruent
values for p_vaddr and p_offset, modulo the page size''. Linux refuses to load binaries that
don't meet this requirement (execve() fails with EINVAL).
This removes the two original implementations in favour of the single
generic one based on the Algorithm type. Previously we had three, very
similar implementations which was somewhat confusing when knowing what
one should actually be used.
The previous polynomials all have equivalent variants available when
using the Algorithm type.
A volume can be mounted as a NTFS path, e.g. as C:\Mnt\Foo. In that case, IOCTL_MOUNTMGR_QUERY_POINTS gives us a mount point with a symlink value something like `\??\Volume{383da0b0-717f-41b6-8c36-00500992b58d}`. In order to get the `C:\Mnt\Foo` path, we can query the mountmgr again using IOCTL_MOUNTMGR_QUERY_DOS_VOLUME_PATH.
Fixes#19731
* Adjust buffer length a bit.
* Fix detecting if file is a script. Logic below was unreachable,
because 99% of scripts failed "At least 255 bytes long" check and were detected as ELF files.
It should be "At least 4" instead (minimum value of "ELF magic length" and "smallest possible interpreter path length").
* Fix parsing interpreter path, when text after shebang:
1. does not have newline,
2. has leading spaces and tabs,
3. separates interpreter and arguments by tab or NUL.
* Remove empty error set from `defaultAbiAndDynamicLinker`.
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
* std.crypto: make ff.ct_unprotected.limbsCmpLt compile
* std.crypto: add ff.ct test
* fix testCt to work on x86
* disable test on stage2-c
---------
Co-authored-by: Frank Denis <124872+jedisct1@users.noreply.github.com>
> Note: This first part is mostly a rephrasing of https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/
> See that article for more details
On Windows, it is possible to execute `.bat`/`.cmd` scripts via CreateProcessW. When this happens, `CreateProcessW` will (under-the-hood) spawn a `cmd.exe` process with the path to the script and the args like so:
cmd.exe /c script.bat arg1 arg2
This is a problem because:
- `cmd.exe` has its own, separate, parsing/escaping rules for arguments
- Environment variables in arguments will be expanded before the `cmd.exe` parsing rules are applied
Together, this means that (1) maliciously constructed arguments can lead to arbitrary command execution via the APIs in `std.process.Child` and (2) escaping according to the rules of `cmd.exe` is not enough on its own.
A basic example argv field that reproduces the vulnerability (this will erroneously spawn `calc.exe`):
.argv = &.{ "test.bat", "\"&calc.exe" },
And one that takes advantage of environment variable expansion to still spawn calc.exe even if the args are properly escaped for `cmd.exe`:
.argv = &.{ "test.bat", "%CMDCMDLINE:~-1%&calc.exe" },
(note: if these spawned e.g. `test.exe` instead of `test.bat`, they wouldn't be vulnerable; it's only `.bat`/`.cmd` scripts that are vulnerable since they go through `cmd.exe`)
Zig allows passing `.bat`/`.cmd` scripts as `argv[0]` via `std.process.Child`, so the Zig API is affected by this vulnerability. Note also that Zig will search `PATH` for `.bat`/`.cmd` scripts, so spawning something like `foo` may end up executing `foo.bat` somewhere in the PATH (the PATH searching of Zig matches the behavior of cmd.exe).
> Side note to keep in mind: On Windows, the extension is significant in terms of how Windows will try to execute the command. If the extension is not `.bat`/`.cmd`, we know that it will not attempt to be executed as a `.bat`/`.cmd` script (and vice versa). This means that we can just look at the extension to know if we are trying to execute a `.bat`/`.cmd` script.
---
This general class of problem has been documented before in 2011 here:
https://learn.microsoft.com/en-us/archive/blogs/twistylittlepassagesallalike/everyone-quotes-command-line-arguments-the-wrong-way
and the course of action it suggests for escaping when executing .bat/.cmd files is:
- Escape first using the non-cmd.exe rules
- Then escape all cmd.exe 'metacharacters' (`(`, `)`, `%`, `!`, `^`, `"`, `<`, `>`, `&`, and `|`) with `^`
However, escaping with ^ on its own is insufficient because it does not stop cmd.exe from expanding environment variables. For example:
```
args.bat %PATH%
```
escaped with ^ (and wrapped in quotes that are also escaped), it *will* stop cmd.exe from expanding `%PATH%`:
```
> args.bat ^"^%PATH^%^"
"%PATH%"
```
but it will still try to expand `%PATH^%`:
```
set PATH^^=123
> args.bat ^"^%PATH^%^"
"123"
```
The goal is to stop *all* environment variable expansion, so this won't work.
Another problem with the ^ approach is that it does not seem to allow all possible command lines to round trip through cmd.exe (as far as I can tell at least).
One known example:
```
args.bat ^"\^"key^=value\^"^"
```
where args.bat is:
```
@echo %1 %2 %3 %4 %5 %6 %7 %8 %9
```
will print
```
"\"key value\""
```
(it will turn the `=` into a space for an unknown reason; other minor variations do roundtrip, e.g. `\^"key^=value\^"`, `^"key^=value^"`, so it's unclear what's going on)
It may actually be possible to escape with ^ such that every possible command line round trips correctly, but it's probably not worth the effort to figure it out, since the suggested mitigation for BatBadBut has better roundtripping and leads to less garbled command lines overall.
---
Ultimately, the mitigation used here is the same as the one suggested in:
https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/
The mitigation steps are reproduced here, noted with one deviation that Zig makes (following Rust's lead):
1. Replace percent sign (%) with %%cd:~,%.
2. Replace the backslash (\) in front of the double quote (") with two backslashes (\\).
3. Replace the double quote (") with two double quotes ("").
4. ~~Remove newline characters (\n).~~
- Instead, `\n`, `\r`, and NUL are disallowed and will trigger `error.InvalidBatchScriptArg` if they are found in `argv`. These three characters do not roundtrip through a `.bat` file and therefore are of dubious/no use. It's unclear to me if `\n` in particular is relevant to the BatBadBut vulnerability (I wasn't able to find a reproduction with \n and the post doesn't mention anything about it except in the suggested mitigation steps); it just seems to act as a 'end of arguments' marker and therefore anything after the `\n` is lost (and same with NUL). `\r` seems to be stripped from the command line arguments when passed through a `.bat`/`.cmd`, so that is also disallowed to ensure that `argv` can always fully roundtrip through `.bat`/`.cmd`.
5. Enclose the argument with double quotes (").
The escaped command line is then run as something like:
cmd.exe /d /e:ON /v:OFF /c "foo.bat arg1 arg2"
Note: Previously, we would pass `foo.bat arg1 arg2` as the command line and the path to `foo.bat` as the app name and let CreateProcessW handle the `cmd.exe` spawning for us, but because we need to pass `/e:ON` and `/v:OFF` to cmd.exe to ensure the mitigation is effective, that is no longer tenable. Instead, we now get the full path to `cmd.exe` and use that as the app name when executing `.bat`/`.cmd` files.
---
A standalone test has also been added that tests two things:
1. Known reproductions of the vulnerability are tested to ensure that they do not reproduce the vulnerability
2. Randomly generated command line arguments roundtrip when passed to a `.bat` file and then are passed from the `.bat` file to a `.exe`. This fuzz test is as thorough as possible--it tests that things like arbitrary Unicode codepoints and unpaired surrogates roundtrip successfully.
Note: In order for the `CreateProcessW` -> `.bat` -> `.exe` roundtripping to succeed, the .exe must split the arguments using the post-2008 C runtime argv splitting implementation, see https://github.com/ziglang/zig/pull/19655 for details on when that change was made in Zig.
ensureTotalCapacityPrecise only satisfies the assumptions made in the ArrayListImpl functions (that there's already enough capacity for the entire converted string if it's all ASCII) when the ArrayList has no items, otherwise it would hit illegal behavior.
Windows does not support RPATH and only searches for DLLs in a small
number of predetermined paths by default, with one of them being the
directory from which the application loaded.
Installing both executables and DLLs to `bin/` by default helps ensure
that the executable can find any DLL artifacts it has linked to.
DLL import libraries are still installed to `lib/`.
These defaults match CMake's behavior.
this patch renames ComptimeStringMap to StaticStringMap, makes it
accept only a single type parameter, and return a known struct type
instead of an anonymous struct. initial motivation for these changes
was to reduce the 'very long type names' issue described here
https://github.com/ziglang/zig/pull/19682.
this breaks the previous API. users will now need to write:
`const map = std.StaticStringMap(T).initComptime(kvs_list);`
* move `kvs_list` param from type param to an `initComptime()` param
* new public methods
* `keys()`, `values()` helpers
* `init(allocator)`, `deinit(allocator)` for runtime data
* `getLongestPrefix(str)`, `getLongestPrefixIndex(str)` - i'm not sure
these belong but have left in for now incase they are deemed useful
* performance notes:
* i posted some benchmarking results here:
https://github.com/travisstaloch/comptime-string-map-revised/issues/1
* i noticed a speedup reducing the size of the struct from 48 to 32
bytes and thus use u32s instead of usize for all length fields
* i noticed speedup storing KVs as a struct of arrays
* latest benchmark shows these wall_time improvements for
debug/safe/small/fast builds: -6.6% / -10.2% / -19.1% / -8.9%. full
output in link above.
Stores the original ref as a query parameter in the URL so that it is
possible to automatically check the upstream if there are any newer
commits.
Also adds a flag which opts-out of the new behaivour, restoring the old.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
