1
Fork 0
Commit Graph

23 Commits

Author SHA1 Message Date
Motiejus Jakštys eced0109ca zig launcher: replace shell wrappers with a binary
Until now we needed to maintain two versions of the zig launcher: one
for Windows and one for everything else. This was problematic for two
reasons:
1. I do not know powershell and thus keep breaking the Windows wrapper
   all the time (see git history of Fabian fixing stuff that I broke).
2. This makes bazel-zig-cc dependent on the system shell, making it not
   really hermetic. So the recently added
   `--experimental_use_hermetic_linux_sandbox` does not work with
   bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`,
   `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`.

Switching to a Zig-based wrapper solves both issues, and we can do this:

    bazel build "$@" \
        --experimental_use_hermetic_linux_sandbox \
        --sandbox_add_mount_pair=/proc \
        <...>

Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to
keep that. I will look into reducing even that dependency separately.

Not all is nice and shiny though: this commit replaces ~80 LOC worth of
shell scripts wrappers with a singe ~300 LOC zig program, which is
arguably harder to understand. However, it is easier to change, at least
for me, because it's a single file with unit tests! Most importantly,
the gnarly code (which resolves paths and sets environment variables) is
cross-platform.

Thanks to Fabian Hahn for testing this on Windows and pointing out
errors.
2023-01-03 11:46:58 +02:00
Jeremy Volkman cc8f113489 Use artifact_name_pattern to specify proper macos and windows artifact names.
On macos, dynamic libraries are generated as "libfoo.dylib".

On windows, executables end with ".exe", static libraries end with ".lib",
and dynamic libraries end with ".dll".
2022-12-18 05:29:04 +02:00
Jeremy Volkman 6ee01496be Use -mcpu=apple_m1 when targeting macos + aarch64 2022-12-18 05:12:24 +02:00
Jeremy Volkman cba7c5fcae Rename _target_darwin to _target_macos
This aligns with bazel's @platforms//os:macos
2022-12-18 05:11:05 +02:00
Motiejus Jakštys 79812dc567 includes: add `libcxxabi`
Otherwise v8 complains.
2022-12-14 00:02:23 +02:00
Motiejus Jakštys 00e126ec85 don't forget libcxx and ar 2022-12-13 17:19:42 +02:00
Motiejus Jakštys 65a8d7c4bb toolchain files: reduce the number of files in the sandbox
We already know which headers are necessary, so it's wasteful to put the
full zig_sdk to every sandbox.

This reduces the number of files in `external/zig_sdk` from 15k to 4k or
6k, depending on the action.
2022-12-13 16:40:03 +02:00
Motiejus Jakštys f30a3afdde remove redundant flags
- -target
- linkopts
2022-10-01 23:27:58 +03:00
Motiejus Jakštys 9b510810cc split all tools to per-codename subdirectories
Go ignores CFLAGS for some commands. That causes unnecessary build of
glibc shims (and libc++.a).

In the GCC days cross-compiler toolchains used to expose a "tool" per
architecture. Let's do the same here. Then Go cannot cheat with skipping
CFLAGS which we normally *always* expect.

The wrapper code is getting gnarly, I know. But it still fits in my head
somehow, so we can still keep adding to it.
2022-10-01 14:35:27 +03:00
Motiejus Jakštys 0302fb630a defs: fix import paths
Empirically these need to come from most specfic to least specific.

The error message is as follows:

    In file included from test/c/main.c:1:
    In file included from external/zig_sdk/lib/libcxx/include/stdio.h:107:
    In file included from external/zig_sdk/lib/libc/include/generic-glibc/stdio.h:38:
    external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:139:3: error:
    # error
      ^
    external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:145:1: error: unknown type name '__STD_TYPE'
    __STD_TYPE __DEV_T_TYPE __dev_t;        /* Type of device numbers.  */

Dissected `generic-glibc/bits/types.h:#error`:

    #if __WORDSIZE == 32
    <...>
    # define __STD_TYPE		__extension__ typedef
    #elif __WORDSIZE == 64
    <...>
    # define __STD_TYPE		typedef
    #else
    # error
    #endif

So we do not have the `__WORDSIZE`. Where does that come from? Probably from a
directory that has an `x86_64` in it. How does that get included? Let's start
with `lib/libcxx/include/stdio.h`:

	16 #include_next <stdio.h>

Now previously our `c++` command line looked like this:
    external/zig_sdk/tools/c++ \
    <...>
    -Iexternal/zig_sdk/lib/include \
    -Iexternal/zig_sdk/lib/libcxx/include \
    -Iexternal/zig_sdk/lib/libcxxabi/include \
    -Iexternal/zig_sdk/lib/libunwind/include \
    -Iexternal/zig_sdk/lib/libc/include/generic-glibc \
    -Iexternal/zig_sdk/lib/libc/include/any-linux-any \
    -Iexternal/zig_sdk/lib/libc/include/x86_64-linux-gnu \
    -Iexternal/zig_sdk/lib/libc/include/x86_64-linux-any \
    -Iexternal/zig_sdk/lib/libc/include/x86-linux-any \
    -Iexternal/zig_sdk/glibc-hacks \
    <...>

So the next place it will find `stdio.h` is in `generic-glibc`, which already
uses the `__WORDSIZE`. If we make the "next" include to be the arch-specific
one instead of the generic-glibc, things start compiling again.

Fix the same fo musl.
2022-09-27 10:15:02 +03:00
Motiejus Jakštys 2f0ae4affa target_musl: remove linkopts
These flags were added in this commit:

    commit 58a04fbfec
    Author: Motiejus Jakštys <motiejus@jakstys.lt>
    Date:   Tue Jun 15 09:10:15 2021 +0300

        bump zig

I am still quite puzzled on why I did this and in such a commit: it
bumps the zig version and changes the linker flag.

Working alone, experimentation time? Probably something like this.
Anyhow, this is a good example of the real value of code reviews.
2022-09-02 13:21:39 +03:00
Motiejus Jakštys 23d9b9d39e [macos] remove libc/include/{}-macos.{}-gnu
This seems to be absent in the Zig SDK for macos targets.

Reported-by: Mike Eastham <meastham@tecton.ai>
2022-08-23 18:12:02 +03:00
Luis Holanda f13e59a30a Add missing include path for MacOS platforms
While integrating the project into our Bazel workspace, we had some
problems while building protoc in an Intel MacBook.

From my testing, the issue seems to be caused by missing include path
in the toolchain configuration to the `-none` variant of the headers.
More details below.

The issue can be reproduced directly inside `bazel-zig-cc` by including
the protobuf workspace as a dependency:

+http_archive(
+    name = "com_google_protobuf",
+    sha256 = "2d9084d3dd13b86ca2e811d2331f780eb86f6d7cb02b405426e3c80dcbfabf25",
+    strip_prefix = "protobuf-3.21.1",
+    url = "https://github.com/protocolbuffers/protobuf/archive/v3.21.1.zip",
+)
+
+load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
+
+protobuf_deps()

And then executing:

bazel build --platforms @zig_sdk//platform:darwin_amd64 @com_google_protobuf//:protoc

The command fails with the same error both in native compilation and
cross-compiling from Linux. The error:

ERROR: <output_base>/external/zlib/BUILD.bazel:37:11: Compiling compress.c failed: undeclared inclusion(s) in rule '@zlib//:zlib':
this rule is missing dependency declarations for the following files included by 'compress.c':
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/cdefs.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_symbol_aliasing.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/limits.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/limits.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/_limits.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/syslimits.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/types.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/types.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/_types.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_types/_int8_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_types/_uintptr_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/_types.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_types.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/machine/endian.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/i386/endian.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/libkern/_OSByteOrder.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/libkern/i386/_OSByteOrder.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_types/_fd_def.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/Availability.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/AvailabilityInternal.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_attr_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_cond_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_condattr_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_rwlock_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_rwlockattr_t.h'
  '<output_base>/external/zig_sdk/lib/libc/include/x86_64-macos.10-none/sys/_pthread/_pthread_t.h'
Target @com_google_protobuf//:protoc failed to build

(<output_base> replaced manually to keep the description sane)

Note that the specific rule included in the error message will vary due
to parallelism, but the error is always the same and the missing paths
always have the same prefix (up to `x86_64-macos.10-none`).

After applying the patch, we confirmed that both native and cross compilations
works as expected and can properly build protoc.

Signed-off-by: Luis Holanda <luiscmholanda@gmail.com>
2022-08-23 18:11:54 +03:00
Motiejus Jakštys b7da83e13e upgrade zig sdk
Also fix darwin zigtarget to none; see 5b813f1a2acdc1668e39008b02a234f3da724552 of zig
2022-06-28 12:47:23 +03:00
Motiejus Jakštys 91ef17196b windows: add dynamic_library_linkopts
The patchset was created before
921fda9260, which added an option to
zig_cc_toolchain_config, causing a "soft merge conflict" after merging.
2022-06-06 10:30:39 +03:00
Fabian Hahn 72475ee012 added support for Windows targets 2022-06-02 05:27:22 +03:00
Fabian Hahn 4a6eef7945 added Windows host support 2022-06-02 05:27:22 +03:00
Jeremy Volkman a0511ca5ee Pass "-Wl,-undefined=dynamic_lookup" for dynamic libraries on darwin.
This linker flag causes undefined symbols to be ignored, which is the
default behavior on linux but not macos. This is required when building
shared libraries to use as e.g. Python modules. This also matches what
Bazel's built-in macos cc toolchain does.
2022-06-01 11:49:13 +03:00
Jeremy Volkman 921fda9260 Adds dynamic_library_linkopts to zig_cc_toolchain_config
If specified, the flags in dynamic_library_linkopts are included in
cpp_link_dynamic_library and cpp_link_nodeps_dynamic_library actions.
2022-06-01 11:49:13 +03:00
Motiejus Jakštys efca6e6428 remove support for glibc autodetection; this is a hermetic toolchain 2022-04-18 11:45:09 +03:00
Motiejus Jakštys 0c02a827ae README prose, the way I understand it 2022-04-18 11:42:13 +03:00
laurynasl 4d65b80903 Add libc constraint and libc aware toolchains
- Also get rid of @bazel_skylib dependency
2022-04-14 14:29:06 +00:00
laurynasl eedfc3312e Move toolchain definitions to toolchains/ 2022-04-07 12:43:34 +03:00