Previously the toolchain wrappers used to assume the zig_lib_dir is
always in "lib". However, it depends on the OS: macos-x86_64 and
linux-aarch64 have it in "lib/zig".
Update the wrapper scripts to reflect that.
Zig does not always share `libc++.a` and the glibc shims. We have observed
to be caused for 2 reasons:
- For some commands Go `chdir`s to `/tmp/`. This causes `ZIG_LIB_DIR` to be
absolute, which blows the cache key to find the right `libc++.a` (because Zig
thinks we are using a different lib dir to compile libc++). This is currently
worked around in `toolchain/defs.bzl` by overfitting to Go and returning
early from that particular invocation.
- Sometimes Bazel's sandbox messes up Zig's cache keys. If one runs without the
sandbox (`--spawn_strategy=standalone`), the cache hit rate and thus the
build time are much better.
This is actively investigated. I am adding `--spawn_strategy=standalone`
to CI to see the speedup that it provides. I will rollback it later.
Go ignores CFLAGS for some commands. That causes unnecessary build of
glibc shims (and libc++.a).
In the GCC days cross-compiler toolchains used to expose a "tool" per
architecture. Let's do the same here. Then Go cannot cheat with skipping
CFLAGS which we normally *always* expect.
The wrapper code is getting gnarly, I know. But it still fits in my head
somehow, so we can still keep adding to it.
'common' is not a thing in Windows; but it needs to be defined.
This makes v1.0.0-rc3 broken for Windows, and not useful (yet) for
anyone else but me. Revert the README update too.
It would "fix" the files on CI and not fail. ci/lint should verify they
are OK first, and give guidance to fix them.
also, `shellcheck -x` is no longer necessary; we can use the plain one
now.
Empirically these need to come from most specfic to least specific.
The error message is as follows:
In file included from test/c/main.c:1:
In file included from external/zig_sdk/lib/libcxx/include/stdio.h:107:
In file included from external/zig_sdk/lib/libc/include/generic-glibc/stdio.h:38:
external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:139:3: error:
# error
^
external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:145:1: error: unknown type name '__STD_TYPE'
__STD_TYPE __DEV_T_TYPE __dev_t; /* Type of device numbers. */
Dissected `generic-glibc/bits/types.h:#error`:
#if __WORDSIZE == 32
<...>
# define __STD_TYPE __extension__ typedef
#elif __WORDSIZE == 64
<...>
# define __STD_TYPE typedef
#else
# error
#endif
So we do not have the `__WORDSIZE`. Where does that come from? Probably from a
directory that has an `x86_64` in it. How does that get included? Let's start
with `lib/libcxx/include/stdio.h`:
16 #include_next <stdio.h>
Now previously our `c++` command line looked like this:
external/zig_sdk/tools/c++ \
<...>
-Iexternal/zig_sdk/lib/include \
-Iexternal/zig_sdk/lib/libcxx/include \
-Iexternal/zig_sdk/lib/libcxxabi/include \
-Iexternal/zig_sdk/lib/libunwind/include \
-Iexternal/zig_sdk/lib/libc/include/generic-glibc \
-Iexternal/zig_sdk/lib/libc/include/any-linux-any \
-Iexternal/zig_sdk/lib/libc/include/x86_64-linux-gnu \
-Iexternal/zig_sdk/lib/libc/include/x86_64-linux-any \
-Iexternal/zig_sdk/lib/libc/include/x86-linux-any \
-Iexternal/zig_sdk/glibc-hacks \
<...>
So the next place it will find `stdio.h` is in `generic-glibc`, which already
uses the `__WORDSIZE`. If we make the "next" include to be the arch-specific
one instead of the generic-glibc, things start compiling again.
Fix the same fo musl.
These flags were added in this commit:
commit 58a04fbfec
Author: Motiejus Jakštys <motiejus@jakstys.lt>
Date: Tue Jun 15 09:10:15 2021 +0300
bump zig
I am still quite puzzled on why I did this and in such a commit: it
bumps the zig version and changes the linker flag.
Working alone, experimentation time? Probably something like this.
Anyhow, this is a good example of the real value of code reviews.
The user may want to build with `--sandbox_tmpfs_path=/tmp`, in which
case the Zig cache will not be shared between sandboxes, massively
slowing down the build. With this feature, the user can pass flags
like `--repo_env=BAZEL_ZIG_CC_CACHE_PREFIX=$HOME/.cache/bazel-zig-cc`
together with `--sandbox_writable_path=$HOME/.cache/bazel-zig-cc` to
put the cache somewhere else which can be shared between sandboxes.
While integrating the project into our Bazel workspace, we had some
problems while building protoc in an Intel MacBook.
From my testing, the issue seems to be caused by missing include path
in the toolchain configuration to the `-none` variant of the headers.
More details below.
The issue can be reproduced directly inside `bazel-zig-cc` by including
the protobuf workspace as a dependency:
+http_archive(
+ name = "com_google_protobuf",
+ sha256 = "2d9084d3dd13b86ca2e811d2331f780eb86f6d7cb02b405426e3c80dcbfabf25",
+ strip_prefix = "protobuf-3.21.1",
+ url = "https://github.com/protocolbuffers/protobuf/archive/v3.21.1.zip",
+)
+
+load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
+
+protobuf_deps()
And then executing:
bazel build --platforms @zig_sdk//platform:darwin_amd64 @com_google_protobuf//:protoc
The command fails with the same error both in native compilation and
cross-compiling from Linux. The error:
ERROR: <output_base>/external/zlib/BUILD.bazel:37:11: Compiling compress.c failed: undeclared inclusion(s) in rule '@zlib//:zlib':
this rule is missing dependency declarations for the following files included by 'compress.c':
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/cdefs.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_symbol_aliasing.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/limits.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/limits.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/_limits.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/syslimits.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/types.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/types.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/_types.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_types/_int8_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_types/_uintptr_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/_types.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_types.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/endian.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/endian.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/libkern/_OSByteOrder.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/libkern/i386/_OSByteOrder.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_types/_fd_def.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/Availability.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/AvailabilityInternal.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_attr_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_cond_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_condattr_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_rwlock_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_rwlockattr_t.h'
'<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_t.h'
Target @com_google_protobuf//:protoc failed to build
(<output_base> replaced manually to keep the description sane)
Note that the specific rule included in the error message will vary due
to parallelism, but the error is always the same and the missing paths
always have the same prefix (up to `x86_64-macos.10-none`).
After applying the patch, we confirmed that both native and cross compilations
works as expected and can properly build protoc.
Signed-off-by: Luis Holanda <luiscmholanda@gmail.com>
Motiejus Jakštys
Motiejus Jakštys