1
Fork 0
Commit Graph

142 Commits

Author SHA1 Message Date
Motiejus Jakštys 1fb93e089c toolchain wrapper: zig_exe path is relative
Just like we know the path to zig_lib_dir, we do know the path to
zig_exe. Lets use that.

This was surfaced by experimenting with
`--experimental_use_hermetic_linux_sandbox`.
`repository_ctx.path("zig")` would return the real file system path to
zig (outside of the sandbox), which is not good when the sandbox is
hermetic.

Co-developed-by: Fabian Hahn <fabian@hahn.graphics>
2022-12-11 18:28:38 +02:00
Fabian Hahn 43b6e1c0c1 fix windows include root path 2022-12-11 18:28:19 +02:00
Motiejus Jakštys 70ef4dd54e remove zig_include_root
It is now consistent across all 5 platforms we support:

    $ for x in *324*.tar.xz; do tar -tf $x | grep libc/ | head -5; done
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    $ for x in *324*.zip; do unzip -l $x | grep libc/ | head -5; done
            0  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
            0  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/
       301018  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.11.tbd
       309693  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.12.tbd
       326079  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.13.tbd
2022-12-07 12:17:44 +02:00
Motiejus Jakštys 6bd588c07b upgrade to zig 0.11.0-dev.324+f61c5f3f5
includes a zig upstream fix for absolute directories: https://github.com/ziglang/zig/issues/13050
2022-11-29 13:51:57 +02:00
Motiejus Jakštys 88e7e47ed2 zig_include_root: fixes
- do not pass where it's unnecessary
- remove trailing slash. That conflicts with ziglang/zig/pull/13596
2022-11-21 06:08:22 +02:00
Motiejus Jakštys dd4239407e lint 2022-11-17 15:47:46 +02:00
Motiejus Jakštys 87fb993408 zig build: add -target automatically 2022-11-17 14:04:25 +02:00
Motiejus Jakštys d195402030 zig build-exe, build-lib, build-obj
This change makes it possible to build zig sources from Bazel rules by
calling the right tool.
2022-11-17 13:51:31 +02:00
Motiejus Jakštys 50103497f6 upgrade zig sdk
The uber1 was broken on x86_64-macos
2022-10-19 11:38:46 +03:00
Motiejus Jakštys ae2746ebeb zig_include_root: support windows too
No real change, just make the logic consistent with Linux/Darwin.

This prepares us for windows-aarch64, which will probably have lib in
lib/zig.
2022-10-13 06:11:25 +03:00
Motiejus Jakštys fed463e1d0 bump zig to 4301+uber1
This is a patched zig release with 13051[1]. While we are waiting for a
resolution on #13050 (which @andrewrk is working on).

[1]: https://patch-diff.githubusercontent.com/raw/ziglang/zig/pull/13051
2022-10-13 05:57:56 +03:00
Motiejus Jakštys e5eff0c911 zig include root: it depends on the OS
Previously the toolchain wrappers used to assume the zig_lib_dir is
always in "lib". However, it depends on the OS: macos-x86_64 and
linux-aarch64 have it in "lib/zig".

Update the wrapper scripts to reflect that.
2022-10-11 14:57:44 +03:00
Motiejus Jakštys 5743865c26 Revert "resolve ZIG_LIB_DIR before passing it to zig"
This reverts commit 3a8c2eecfe.

Causes the dependency paths to be absolute.
2022-10-05 14:48:19 +03:00
Motiejus Jakštys 3a8c2eecfe resolve ZIG_LIB_DIR before passing it to zig
That way the zig's cache keys will be the same for all sandboxes.

Also see https://github.com/ziglang/zig/issues/13050#issuecomment-1268074032
2022-10-05 14:24:19 +03:00
Motiejus Jakštys 50ca49181f Zig cache hacks/optimizations
Zig does not always share `libc++.a` and the glibc shims. We have observed
to be caused for 2 reasons:

- For some commands Go `chdir`s to `/tmp/`. This causes `ZIG_LIB_DIR` to be
  absolute, which blows the cache key to find the right `libc++.a` (because Zig
  thinks we are using a different lib dir to compile libc++). This is currently
  worked around in `toolchain/defs.bzl` by overfitting to Go and returning
  early from that particular invocation.
- Sometimes Bazel's sandbox messes up Zig's cache keys. If one runs without the
  sandbox (`--spawn_strategy=standalone`), the cache hit rate and thus the
  build time are much better.

This is actively investigated. I am adding `--spawn_strategy=standalone`
to CI to see the speedup that it provides. I will rollback it later.
2022-10-03 05:50:53 +03:00
Motiejus Jakštys b1209d0165 gohack only in tools/<arch>/c++ 2022-10-01 18:53:38 +03:00
Motiejus Jakštys 8919c5b703 fix tool paths for different OSes/environments 2022-10-01 14:39:59 +03:00
Motiejus Jakštys 9b510810cc split all tools to per-codename subdirectories
Go ignores CFLAGS for some commands. That causes unnecessary build of
glibc shims (and libc++.a).

In the GCC days cross-compiler toolchains used to expose a "tool" per
architecture. Let's do the same here. Then Go cannot cheat with skipping
CFLAGS which we normally *always* expect.

The wrapper code is getting gnarly, I know. But it still fits in my head
somehow, so we can still keep adding to it.
2022-10-01 14:35:27 +03:00
Motiejus Jakštys e96f5bb59c lint 2022-10-01 05:31:50 +03:00
Motiejus Jakštys e9638443c6 tools wrapper: fix for windows
'common' is not a thing in Windows; but it needs to be defined.

This makes v1.0.0-rc3 broken for Windows, and not useful (yet) for
anyone else but me. Revert the README update too.
2022-10-01 05:12:45 +03:00
Motiejus Jakštys c4ab1cfaea buildifier 2022-09-30 15:17:49 +03:00
Motiejus Jakštys 9e8d89b40d CGo-specific: reduce unneeded compilation
This adds an ugly workaround for
https://go-review.googlesource.com/c/go/+/436884 ; fingers crossed.

Impact: ~20-25s reduced first build time for a particular
architecture+glibc.
2022-09-30 14:24:56 +03:00
Fabian Hahn 04b95526ae fix Windows host support with relative paths 2022-09-29 11:32:19 +03:00
Motiejus Jakštys 08b02cb7d2 rename absolute_path to zig_path
this is not an absolute path.
2022-09-27 15:37:28 +03:00
Motiejus Jakštys 0302fb630a defs: fix import paths
Empirically these need to come from most specfic to least specific.

The error message is as follows:

    In file included from test/c/main.c:1:
    In file included from external/zig_sdk/lib/libcxx/include/stdio.h:107:
    In file included from external/zig_sdk/lib/libc/include/generic-glibc/stdio.h:38:
    external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:139:3: error:
    # error
      ^
    external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:145:1: error: unknown type name '__STD_TYPE'
    __STD_TYPE __DEV_T_TYPE __dev_t;        /* Type of device numbers.  */

Dissected `generic-glibc/bits/types.h:#error`:

    #if __WORDSIZE == 32
    <...>
    # define __STD_TYPE		__extension__ typedef
    #elif __WORDSIZE == 64
    <...>
    # define __STD_TYPE		typedef
    #else
    # error
    #endif

So we do not have the `__WORDSIZE`. Where does that come from? Probably from a
directory that has an `x86_64` in it. How does that get included? Let's start
with `lib/libcxx/include/stdio.h`:

	16 #include_next <stdio.h>

Now previously our `c++` command line looked like this:
    external/zig_sdk/tools/c++ \
    <...>
    -Iexternal/zig_sdk/lib/include \
    -Iexternal/zig_sdk/lib/libcxx/include \
    -Iexternal/zig_sdk/lib/libcxxabi/include \
    -Iexternal/zig_sdk/lib/libunwind/include \
    -Iexternal/zig_sdk/lib/libc/include/generic-glibc \
    -Iexternal/zig_sdk/lib/libc/include/any-linux-any \
    -Iexternal/zig_sdk/lib/libc/include/x86_64-linux-gnu \
    -Iexternal/zig_sdk/lib/libc/include/x86_64-linux-any \
    -Iexternal/zig_sdk/lib/libc/include/x86-linux-any \
    -Iexternal/zig_sdk/glibc-hacks \
    <...>

So the next place it will find `stdio.h` is in `generic-glibc`, which already
uses the `__WORDSIZE`. If we make the "next" include to be the arch-specific
one instead of the generic-glibc, things start compiling again.

Fix the same fo musl.
2022-09-27 10:15:02 +03:00
Ken Micklas ab59c18d60 WIP: use relative paths for tools 2022-09-27 10:14:59 +03:00
Motiejus Jakštys f6b16c386f upgrade to zig 0.10.0-dev.4166+cae76d829
- llvm 15
- uses less space for /tmp/bazel-zig-cc
2022-09-25 13:51:59 +03:00
Motiejus Jakštys 026d234bc2 Revert "bump to zig 0.10.0-dev.3999+fda6d4477"
This reverts commit f161caf870.

This version has a bug that's been fixed upstream, but a new nightly did
not come out yet: https://github.com/ziglang/zig/issues/12858
2022-09-22 11:57:39 +03:00
Ken Micklas 64b6c702db Fix include root path for macos-aarch64 2022-09-15 22:19:06 +03:00
Ken Micklas ba0b0376ea Move Label dependencies above archive fetching to reduce restarts 2022-09-15 22:18:43 +03:00
Motiejus Jakštys f161caf870 bump to zig 0.10.0-dev.3999+fda6d4477
llvm 15
2022-09-14 11:54:04 +03:00
Motiejus Jakštys bf32b5ee92 repository_rule: depend on $BAZEL_ZIG_CC_CACHE_PREFIX
Otherwise the repository doesn't get re-downloaded/wiped when this
parameter changes.
2022-09-12 13:46:07 +03:00
Motiejus Jakštys e1d699f7ed upgrade to 0.10.0-dev.3838+77f31ebbb 2022-09-01 09:28:43 +03:00
Motiejus Jakštys 79674a1d96 replace /usr/bin/env sh with /bin/sh
/bin/sh exists in NixOS and in fact is the only file in `/bin`:

    $ docker run -ti --rm nixos/nix ls /bin /usr/bin
    /bin:
    sh

    /usr/bin:
    env

More context in
https://lists.sr.ht/~motiejus/bazel-zig-cc/%3CCAFVMu-pvNx%2BpeQYdge_fvmSGrHDmn78VmoYwTxkDgMjbzfVAwQ%40mail.gmail.com%3E
2022-08-30 09:21:27 +03:00
Motiejus Jakštys 342b239bb9 nit: style of cache prefix definition 2022-08-29 14:44:38 +03:00
Motiejus Jakštys e861c5b4be cache_prefix: clean up OS logic
'Windows' and 'Other' and guessing of cache prefix was convoluted
2022-08-29 13:01:09 +03:00
Fabian Hahn a410141ca4 add cache prefix configuration for Windows 2022-08-28 20:18:21 +03:00
Motiejus Jakštys 15f9578591 use a specific template if cache prefix is known 2022-08-26 10:55:29 +03:00
Ken Micklas 0612c6ed9c Support configuring cache prefix with an environment variable
The user may want to build with `--sandbox_tmpfs_path=/tmp`, in which
case the Zig cache will not be shared between sandboxes, massively
slowing down the build. With this feature, the user can pass flags
like `--repo_env=BAZEL_ZIG_CC_CACHE_PREFIX=$HOME/.cache/bazel-zig-cc`
together with `--sandbox_writable_path=$HOME/.cache/bazel-zig-cc` to
put the cache somewhere else which can be shared between sandboxes.
2022-08-26 10:47:25 +03:00
Motiejus Jakštys db7beefbe4 c++: fall back to cache dir in /tmp
rules_go is making calls that do `env - <...>` and are setting neither
TMPDIR nor HOME. So we should always fallback somewhere.
2022-08-12 09:25:37 +03:00
Motiejus Jakštys e50d1597f4 upgrade zig sdk to one of last night 2022-08-12 05:38:34 +03:00
Luis Holanda c4125cf13b Support NixOS and others non-FHS Linux distros
As it expects Bash to be at `/bin/bash`, the current implementation
fails to execute on NixOS, given that there bash is in a non-standard
path (i.g. `/nix/store/<hash>-bash-interactive-<version>/bin/bash`).

This patch specifically changes `/bin/bash` paths to use `/usr/bin/env bash`,
which should give the correct path in every Unix system.

Signed-off-by: Luis Holanda <luiscmholanda@gmail.com>
2022-07-28 12:44:57 -07:00
Motiejus Jakštys df11a2af38 bump zig to 0.10.0-dev.2977+7d2e14267"
changes:
- upgrade from llvm 13 to llvm 14
- add --compress-debug-sections=zlib
2022-07-12 14:09:11 +03:00
Motiejus Jakštys 0818e20167 bump zig upstream 2022-07-04 15:39:41 +03:00
Motiejus Jakštys b7da83e13e upgrade zig sdk
Also fix darwin zigtarget to none; see 5b813f1a2acdc1668e39008b02a234f3da724552 of zig
2022-06-28 12:47:23 +03:00
Motiejus Jakštys 86ae317685 zig sdk: rename {ext} to {_ext}
Also update README and release scripts
2022-06-06 10:32:42 +03:00
Fabian Hahn 4a6eef7945 added Windows host support 2022-06-02 05:27:22 +03:00
Motiejus Jakštys 8fec0e3441 zig sdk: download from zig upstream as well as jakstys.lt
Now that the toolchain supports multiple mirrors, we can have more
robust defaults: try ziglang.org first, then dl.jakstys.lt when that one
removes the build we rely on.

This will make builds less prone to inevitable failures of my home
server.
2022-05-18 10:42:51 +03:00
Motiejus Jakštys 0044260206 upgrade zig to 0.10.0-dev.2252+a4369918b 2022-05-17 14:41:02 +03:00
Motiejus Jakštys 4fa5eff207 [linux-aarch64] fix top-level include path 2022-05-06 10:46:32 +03:00