Subject: [PATCH bazel-zig-cc 0/3] Windows support

From: Fabian Hahn <[email-masked]>
Date: Fri, 20 May 2022 16:02:29 +0100

Hello there,

First of all, thank you for this great project! I was always intrigued
by zig's C/C++ compilation abilities but wasn't too keen on adopting it
as a build system. Being able to use zig as a bazel toolchain using this
project is incredible and exactly the way I always wanted to use zig.

My intended use case for bazel-zig-cc was to enable cross compilation
between the two supported platforms Linux and Windows in my project [1].
To this end, I started looking into extending bazel-zig-cc with Windows
support both as a host and target system. The attached three commits in
made this work and successfully allowed me to cross compile all of my
cc_binary targets between Linux and Windows using zig cc. That said,
they do not actually depend on each other and can be used individually.
I hope others might find them useful, too, and would like to ask you to
consider accepting them into the mainline of the project.

Best,
Fabian

[1] https://github.com/FabianHahn/shoveler

Fabian Hahn (3):
  added windows host support
  added support for Windows targets
  added platform constraint setting for selecting if zig cc is used

 toolchain/BUILD.sdk.bazel           |  1 +
 toolchain/defs.bzl                  | 58 +++++++++++++++++++++++------
 toolchain/platform/BUILD            | 15 ++++++++
 toolchain/platform/defs.bzl         |  8 +++-
 toolchain/private/BUILD.sdk.bazel   |  1 +
 toolchain/private/cc_toolchains.bzl |  6 +--
 toolchain/private/defs.bzl          | 27 +++++++++++++-
 7 files changed, 99 insertions(+), 17 deletions(-)

-- 
2.17.1

---

From: Fabian Hahn <[email-masked]>
Date: Fri, 20 May 2022 16:02:30 +0100

---
This commit adds support for Windows as a host system. For example, given
an existing bazel project on Windows, this commit enables cross
compilation to other supported target platforms of bazel-zig-cc such as
Linux. I've successfully used it to cross compile a Linux cc_binary from a
Windows machine and validated that it works correctly by copying it to a
Linux box and launching it there.

Apart from downloading the correct zig SDK version on Windows, the main
crux here was to actually make all of the zig tools properly executable.
In order for the wrapper script to be executable, it needs to be a batch
script with the proper extension instead of bash. Further, the actual
zig binary itself is called zig.exe instead of just zig on Windows.

 toolchain/BUILD.sdk.bazel           |  1 +
 toolchain/defs.bzl                  | 58 +++++++++++++++++++++++------
 toolchain/private/BUILD.sdk.bazel   |  1 +
 toolchain/private/cc_toolchains.bzl |  6 +--
 toolchain/private/defs.bzl          |  8 +++-
 5 files changed, 58 insertions(+), 16 deletions(-)

diff --git a/toolchain/BUILD.sdk.bazel b/toolchain/BUILD.sdk.bazel
index ada3e5d..e1505b1 100644
--- a/toolchain/BUILD.sdk.bazel
+++ b/toolchain/BUILD.sdk.bazel
@@ -7,6 +7,7 @@ package(

 declare_files(
+    os = {os},
     zig_include_root = {zig_include_root},
 )
 
diff --git a/toolchain/defs.bzl b/toolchain/defs.bzl
index 9ab4e88..d334aca 100644
--- a/toolchain/defs.bzl
+++ b/toolchain/defs.bzl
@@ -1,6 +1,6 @@
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("@bazel_tools//tools/build_defs/repo:utils.bzl", "read_user_netrc", "use_netrc")
-load("@bazel-zig-cc//toolchain/private:defs.bzl", "DEFAULT_INCLUDE_DIRECTORIES", "ZIG_TOOL_PATH", "target_structs")
+load("@bazel-zig-cc//toolchain/private:defs.bzl", "DEFAULT_INCLUDE_DIRECTORIES", "zig_tool_path", "target_structs")
 
 _fcntl_map = """
 GLIBC_2.2.5 {
@@ -16,21 +16,21 @@ __asm__(".symver fcntl64, [email-masked]");
 """
 
 # Official recommended version. Should use this when we have a usable release.
-URL_FORMAT_RELEASE = "https://ziglang.org/download/{version}/zig-{host_platform}-{version}.tar.xz"
+URL_FORMAT_RELEASE = "https://ziglang.org/download/{version}/zig-{host_platform}-{version}.{ext}"
 
 # Caution: nightly releases are purged from ziglang.org after ~90 days. A real
 # solution would be to allow the downstream project specify their own mirrors.
 # This is explained in
 # https://sr.ht/~motiejus/bazel-zig-cc/#alternative-download-urls and is
 # awaiting my attention or your contribution.
-URL_FORMAT_NIGHTLY = "https://ziglang.org/builds/zig-{host_platform}-{version}.tar.xz"
+URL_FORMAT_NIGHTLY = "https://ziglang.org/builds/zig-{host_platform}-{version}.{ext}"
 
 # Author's mirror that doesn't purge the nightlies so aggressively. I will be
 # cleaning those up manually only after the artifacts are not in use for many
 # months in bazel-zig-cc. dl.jakstys.lt is a small x86_64 server with an NVMe
 # drive sitting in my home closet on a 1GB/s symmetric residential connection,
 # which, as of writing, has been quite reliable.
-URL_FORMAT_JAKSTYS = "https://dl.jakstys.lt/zig/zig-{host_platform}-{version}.tar.xz"
+URL_FORMAT_JAKSTYS = "https://dl.jakstys.lt/zig/zig-{host_platform}-{version}.{ext}"
 
 _VERSION = "0.10.0-dev.2252+a4369918b"
 
@@ -39,12 +39,22 @@ _HOST_PLATFORM_SHA256 = {
     "linux-x86_64": "1d3c3769eba85a4334c93a3cfa35ad0ef914dd8cf9fd502802004c6908f5370c",
     "macos-aarch64": "ab46e7499e5bd7b6d6ff2ac331e1a4aa875a01b270dc40306bc29dbaf216fccf",
     "macos-x86_64": "fb213f996bcab805839e401292c42a92b63cd97deb1631e31bd61f534b7f6b1c",
+    "windows-x86_64": "14e43a64026512161f3d6201d8972a28f0508da2782c16e980f2ffa3bb7e6720",
+}
+
+_HOST_PLATFORM_EXT = {
+    "linux-aarch64": "tar.xz",
+    "linux-x86_64": "tar.xz",
+    "macos-aarch64": "tar.xz",
+    "macos-x86_64": "tar.xz",
+    "windows-x86_64": "zip",
 }
 
 def toolchains(
         version = _VERSION,
         url_formats = [URL_FORMAT_NIGHTLY, URL_FORMAT_JAKSTYS],
-        host_platform_sha256 = _HOST_PLATFORM_SHA256):
+        host_platform_sha256 = _HOST_PLATFORM_SHA256,
+        host_platform_ext = _HOST_PLATFORM_EXT):
     """
         Download zig toolchain and declare bazel toolchains.
         The platforms are not registered automatically, that should be done by
@@ -56,11 +66,13 @@ def toolchains(
         version = version,
         url_formats = url_formats,
         host_platform_sha256 = host_platform_sha256,
+        host_platform_ext = host_platform_ext,
         host_platform_include_root = {
             "linux-aarch64": "lib/zig/",
             "linux-x86_64": "lib/",
             "macos-aarch64": "lib/zig/",
             "macos-x86_64": "lib/zig/",
+            "windows-x86_64": "lib/",
         },
     )
 
@@ -81,6 +93,10 @@ export ZIG_GLOBAL_CACHE_DIR=$ZIG_LOCAL_CACHE_DIR
 exec "{zig}" "{zig_tool}" "$@"
 """
 
+ZIG_TOOL_WRAPPER_WINDOWS = """@echo off
+"{zig}" "{zig_tool}" %*
+"""
+
 _ZIG_TOOLS = [
     "c++",
     "cc",
@@ -103,11 +119,16 @@ def _zig_repository_impl(repository_ctx):
     if os.startswith("mac os"):
         os = "macos"
 
+    if os.startswith("windows"):
+        os = "windows"
+
     host_platform = "{}-{}".format(os, arch)
 
     zig_include_root = repository_ctx.attr.host_platform_include_root[host_platform]
     zig_sha256 = repository_ctx.attr.host_platform_sha256[host_platform]
+    zig_ext = repository_ctx.attr.host_platform_ext[host_platform]
     format_vars = {
+        "ext": zig_ext,
         "version": repository_ctx.attr.version,
         "host_platform": host_platform,
     }
@@ -121,12 +142,19 @@ def _zig_repository_impl(repository_ctx):
     )
 
     for zig_tool in _ZIG_TOOLS:
-        repository_ctx.file(
-            ZIG_TOOL_PATH.format(zig_tool = zig_tool),
-            ZIG_TOOL_WRAPPER.format(
-                zig = str(repository_ctx.path("zig")),
+        zig_tool_wrapper = ZIG_TOOL_WRAPPER.format(
+            zig = str(repository_ctx.path("zig")),
+            zig_tool = zig_tool,
+        )
+        if os == "windows":
+            zig_tool_wrapper = ZIG_TOOL_WRAPPER_WINDOWS.format(
+                zig = str(repository_ctx.path("zig")).replace('/', '\\') + ".exe",
                 zig_tool = zig_tool,
-            ),
+            )
+
+        repository_ctx.file(
+            zig_tool_path(os).format(zig_tool = zig_tool),
+            zig_tool_wrapper,
         )
 
     repository_ctx.file(
@@ -157,6 +185,7 @@ def _zig_repository_impl(repository_ctx):
             executable = False,
             substitutions = {
                 "{absolute_path}": _quote(str(repository_ctx.path(""))),
+                "{os}": _quote(os),
                 "{zig_include_root}": _quote(zig_include_root),
             },
         )
@@ -167,6 +196,7 @@ zig_repository = repository_rule(
         "host_platform_sha256": attr.string_dict(),
         "url_formats": attr.string_list(allow_empty = False),
         "host_platform_include_root": attr.string_dict(),
+        "host_platform_ext": attr.string_dict(),
     },
     implementation = _zig_repository_impl,
 )
@@ -175,9 +205,13 @@ def filegroup(name, **kwargs):
     native.filegroup(name = name, **kwargs)
     return ":" + name
 
-def declare_files(zig_include_root):
+def declare_files(os, zig_include_root):
     filegroup(name = "empty")
-    native.exports_files(["zig"], visibility = ["//visibility:public"])
+    if os == "windows":
+        native.exports_files(["zig.exe"], visibility = ["//visibility:public"])
+        native.alias(name = "zig", actual = ":zig.exe")
+    else:
+        native.exports_files(["zig"], visibility = ["//visibility:public"])
     filegroup(name = "lib/std", srcs = native.glob(["lib/std/**"]))
 
     lazy_filegroups = {}
diff --git a/toolchain/private/BUILD.sdk.bazel b/toolchain/private/BUILD.sdk.bazel
index 39a6a93..3e7c23e 100644
--- a/toolchain/private/BUILD.sdk.bazel
+++ b/toolchain/private/BUILD.sdk.bazel
@@ -1,6 +1,7 @@
 load("@bazel-zig-cc//toolchain/private:cc_toolchains.bzl", "declare_cc_toolchains")
 
 declare_cc_toolchains(
+    os = {os},
     absolute_path = {absolute_path},
     zig_include_root = {zig_include_root},
 )
diff --git a/toolchain/private/cc_toolchains.bzl b/toolchain/private/cc_toolchains.bzl
index 64e99f1..a0c7da0 100644
--- a/toolchain/private/cc_toolchains.bzl
+++ b/toolchain/private/cc_toolchains.bzl
@@ -1,4 +1,4 @@
-load(":defs.bzl", "DEFAULT_INCLUDE_DIRECTORIES", "ZIG_TOOL_PATH", "target_structs")
+load(":defs.bzl", "DEFAULT_INCLUDE_DIRECTORIES", "zig_tool_path", "target_structs")
 load("@bazel-zig-cc//toolchain:zig_toolchain.bzl", "zig_cc_toolchain_config")
 
 DEFAULT_TOOL_PATHS = {
@@ -11,7 +11,7 @@ DEFAULT_TOOL_PATHS = {
     "strip": "/usr/bin/false",
 }.items()
 
-def declare_cc_toolchains(absolute_path, zig_include_root):
+def declare_cc_toolchains(os, absolute_path, zig_include_root):
     for target_config in target_structs():
         gotarget = target_config.gotarget
         zigtarget = target_config.zigtarget
@@ -28,7 +28,7 @@ def declare_cc_toolchains(absolute_path, zig_include_root):
             if path[0] == "/":
                 absolute_tool_paths[name] = path
                 continue
-            tool_path = ZIG_TOOL_PATH.format(zig_tool = path)
+            tool_path = zig_tool_path(os).format(zig_tool = path)
             absolute_tool_paths[name] = "%s/%s" % (absolute_path, tool_path)
 
         linkopts = target_config.linkopts
diff --git a/toolchain/private/defs.bzl b/toolchain/private/defs.bzl
index 6059065..6804104 100644
--- a/toolchain/private/defs.bzl
+++ b/toolchain/private/defs.bzl
@@ -4,7 +4,7 @@ DEFAULT_INCLUDE_DIRECTORIES = [
     "libcxxabi/include",
 ]
 
-ZIG_TOOL_PATH = "tools/{zig_tool}"
+_ZIG_TOOL_PATH = "tools/{zig_tool}"
 
 # Zig supports even older glibcs than defined below, but we have tested only
 # down to 2.17.
@@ -30,6 +30,12 @@ _GLIBCS = [
 
 LIBCS = ["musl"] + ["gnu.{}".format(glibc) for glibc in _GLIBCS]
 
+def zig_tool_path(os):
+    if os == "windows":
+        return _ZIG_TOOL_PATH + ".bat"
+    else:
+        return _ZIG_TOOL_PATH
+
 def target_structs():
     ret = []
     for zigcpu, gocpu in (("x86_64", "amd64"), ("aarch64", "arm64")):
-- 
2.17.1

---

From: Fabian Hahn <[email-masked]>
Date: Fri, 20 May 2022 16:02:31 +0100

---
This commit adds support for Windows targets to bazel-zig-cc by
registering the appropriate toolchains and platforms for it. I've
validated this by building a cc_binary on Linux targeting
the windows_amd64 platform/toolchain combination, copying the resulting
binary to a Windows box and successfully executing it there. I was also
able to run the resulting binary under wine.

One thing to note is that even though cross compilation builds work
completely fine, the resulting build outputs are named .a for static
libraries (instead of .lib), .so for dynamic libraries (instead of
.dll), and have no extension for executables (instead of .exe). I tried
to find out why this is but unfortunately couldn't figure it out. That
said, the resulting binaries work completely fine.

 toolchain/platform/defs.bzl |  7 ++++++-
 toolchain/private/defs.bzl  | 19 +++++++++++++++++++
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/toolchain/platform/defs.bzl b/toolchain/platform/defs.bzl
index cf42191..07e69a6 100644
--- a/toolchain/platform/defs.bzl
+++ b/toolchain/platform/defs.bzl
@@ -1,11 +1,16 @@
 load("@bazel-zig-cc//toolchain/private:defs.bzl", "LIBCS")
 
 _CPUS = (("x86_64", "amd64"), ("aarch64", "arm64"))
+_OS = {
+    "linux": ["linux"],
+    "macos": ["macos", "darwin"],
+    "windows": ["windows"],
+}
 
 def declare_platforms():
     # create @zig_sdk//{os}_{arch}_platform entries with zig and go conventions
     for zigcpu, gocpu in _CPUS:
-        for bzlos, oss in {"linux": ["linux"], "macos": ["macos", "darwin"]}.items():
+        for bzlos, oss in _OS.items():
             for os in oss:
                 declare_platform(gocpu, zigcpu, bzlos, os)
 
diff --git a/toolchain/private/defs.bzl b/toolchain/private/defs.bzl
index 6804104..2ba61f7 100644
--- a/toolchain/private/defs.bzl
+++ b/toolchain/private/defs.bzl
@@ -40,6 +40,7 @@ def target_structs():
     ret = []
     for zigcpu, gocpu in (("x86_64", "amd64"), ("aarch64", "arm64")):
         ret.append(_target_darwin(gocpu, zigcpu))
+        ret.append(_target_windows(gocpu, zigcpu))
         ret.append(_target_linux_musl(gocpu, zigcpu))
         for glibc in _GLIBCS:
             ret.append(_target_linux_gnu(gocpu, zigcpu, glibc))
@@ -69,6 +70,24 @@ def _target_darwin(gocpu, zigcpu):
         tool_paths = {"ld": "ld64.lld"},
     )
 
+def _target_windows(gocpu, zigcpu):
+    return struct(
+        gotarget = "windows_{}".format(gocpu),
+        zigtarget = "{}-windows-gnu".format(zigcpu),
+        includes = [
+            "libunwind/include",
+            "libc/include/any-windows-any",
+        ],
+        linkopts = [],
+        copts = [],
+        bazel_target_cpu = "x64_windows",
+        constraint_values = [
+            "@platforms//os:windows",
+            "@platforms//cpu:{}".format(zigcpu),
+        ],
+        tool_paths = {"ld": "ld64.lld"},
+    )
+
 def _target_linux_gnu(gocpu, zigcpu, glibc_version):
     glibc_suffix = "gnu.{}".format(glibc_version)
 
-- 
2.17.1

---

From: Fabian Hahn <[email-masked]>
Date: Fri, 20 May 2022 16:02:32 +0100

---
This commit adds a constraint_setting that can be used to select()
between builds with and without zig cc. The reason I added this is
because I found that selecting on @bazel_tools//platforms:windows is not
enough for Windows builds since that constraint value alone will match
both native builds using the MSVC compiler as well as builds using zig
cc. The problem arises when passing custom compiler on linker flags that
differ in syntax between MSVC and zig's clang format. For example, to
link the gdi32 system library on Windows MSVC expects the argument
"-DEFAULTLIB:gdi32.lib". However, with zig cc this needs to be "-lgdi32"
instead. Having the additional constraint setting for the bazel-zig-cc
platforms allows you to define more specific config settings such as:

config_setting(
    name = "windows",
    constraint_values = [
        "@bazel_tools//platforms:windows",
        "@zig_sdk//platform:zig_cc_disabled",
    ],
)

config_setting(
    name = "windows_zig",
    constraint_values = [
        "@bazel_tools//platforms:windows",
        "@zig_sdk//platform:zig_cc_enabled",
    ],
)

This can then be used to switch between linker options as follows:
cc_binary(
    # [...]
    linkopts = select({
        ":windows": ["-DEFAULTLIB:gdi32.lib"],
        ":windows_zig": ["-lgdi32"],
    }),
)

 toolchain/platform/BUILD    | 15 +++++++++++++++
 toolchain/platform/defs.bzl |  1 +
 2 files changed, 16 insertions(+)

diff --git a/toolchain/platform/BUILD b/toolchain/platform/BUILD
index f43b852..807ac3c 100644
--- a/toolchain/platform/BUILD
+++ b/toolchain/platform/BUILD
@@ -4,4 +4,19 @@ package(
     default_visibility = ["//visibility:public"],
 )
 
+constraint_setting(
+    name = "using_zig_cc",
+    default_constraint_value = "zig_cc_disabled",
+)
+
+constraint_value(
+    name = "zig_cc_disabled",
+    constraint_setting = "using_zig_cc",
+)
+
+constraint_value(
+    name = "zig_cc_enabled",
+    constraint_setting = "using_zig_cc",
+)
+
 declare_platforms()
diff --git a/toolchain/platform/defs.bzl b/toolchain/platform/defs.bzl
index 07e69a6..05c0744 100644
--- a/toolchain/platform/defs.bzl
+++ b/toolchain/platform/defs.bzl
@@ -32,6 +32,7 @@ def declare_platform(gocpu, zigcpu, bzlos, os, suffix = "", extra_constraints =
     constraint_values = [
         "@platforms//os:{}".format(bzlos),
         "@platforms//cpu:{}".format(zigcpu),
+        "@zig_sdk//platform:zig_cc_enabled",
     ] + extra_constraints
 
     native.platform(
-- 
2.17.1

---

From: "builds.sr.ht" <[email-masked]>
Date: Fri, 20 May 2022 15:16:31 +0000

bazel-zig-cc/patches/.build.yml: FAILED in 13m9s

[Windows support][0] from [Fabian Hahn][1]

[0]: https://lists.sr.ht/~motiejus/bazel-zig-cc/patches/32423
[1]: mailto:[email-masked]

✗ #762646 FAILED bazel-zig-cc/patches/.build.yml https://builds.sr.ht/~motiejus/job/762646

---

From: Motiejus Jakštys <[email-masked]>
Date: Sat, 21 May 2022 15:56:28 +0300

Hi Fabian,

On Fri, May 20, 2022 at 04:02:29PM +0100, Fabian Hahn wrote:
> support both as a host and target system.

Happy to see you are finding it useful!

> The attached three commits in

I don't mind Windows support, as long as there is somebody to
test and maintain it. Once we land this, that will be you. :)

Since none of the bazel-zig-cc contributors are using Windows for
development, we don't have means to test if we did not break something.
So, until we can have a Windows CI, you will be on the hook to test it
with new bazel-zig-cc releases.

And until we are no sr.ht, there is no chance we will have Windows CI.
Unless it can be run under Wine? Do you have appetite to try?

>   added support for Windows targets

The above look reasonable.

>   added platform constraint setting for selecting if zig cc is used

I am not sure about this. Laurynai, can you have a peek if this can't be
expressed via a platform instead? This setting doesn't look right from a
quick glimpse.

Motiejus

---

From: Laurynas Lubys <[email-masked]>
Date: Mon, 23 May 2022 19:06:16 +0300

Hi Fabian,

While I see your use case, I think the solution you outlined is a bit
too invasive for the problem you raised. I would prefer bazel-zig-cc
to only deal with terms related to zig-cc and not other compilers.

Here's how you could solve this bit without changes in bazel-zig-cc:
1. Define the `constraint_value`s in your project
2. Define a custom platform (e.g. windows_msvc and windows_zig),
specifying the bazel-zig-cc platform in the `parents` field and adding
the `constraint_values` from step 1 accordingly.
3. Use your custom platforms instead of the ones from bazel-zig-cc directly.

Cheers,
Laurynas

---

From: [email-masked]
Date: Mon, 23 May 2022 20:25:05 +0100

Hi Motiejus,

> with new bazel-zig-cc releases.

I'm happy to try doing this. I'm definitely interested in keeping my own
project up to date with new releases of the rules too, which should
provide some implicit coverage.

> Unless it can be run under Wine? Do you have appetite to try?

I will look into this. I'm pretty confident that the Windows target
support (second patch) can be covered by wine testing. I'm more 
sceptical
about Windows host support, since you'd have to run Windows bazel itself
under Wine which I'm not sure is supported and might not work.

> expressed via a platform instead? This setting doesn't look right from 

I'll look into Laurynai's suggestion of using a custom platform for 
this.
I'm by no means a bazel expert so this patch was the best I could get
working, and I'm very happy to try a better solution if there is one.

I'll submit a second version of the patch set once I've done this. There
is also a linter failure that I'm going to address.

Best,
Fabian

---

From: [email-masked]
Date: Mon, 23 May 2022 20:28:32 +0100

Hi Laurynas,

> 3. Use your custom platforms instead of the ones from bazel-zig-cc 

Thanks a lot for the pointers, I didn't realize that defining a custom
platform could help here. I'll try this.

Best,
Fabian

---

From: Motiejus Jakštys <[email-masked]>
Date: Tue, 24 May 2022 10:54:34 +0300

On Mon, May 23, 2022 at 08:25:05PM +0100, [email-masked] wrote:
> support (second patch) can be covered by wine testing. I'm more sceptical

Out of curiosity I applied your patches, downloaded bazelisk.exe, and
tried to run it via wine, to be greeted with:

    $ wine ./bin/bazelisk.exe query ...
    ERROR: SymlinkDirectories(C:/users/motiejus/_bazel_motiejus/install/5f8090d9a86a5bdc9afbbd92e4f1b167,
        c:\users\motiejus\_bazel_motiejus\n3lawf4n\install): CreateJunction: 
    FATAL: failed to create installation symlink 'c:\users\motiejus\_bazel_motiejus\n3lawf4n\install': success

... And no obvious suggestions from my favorite search engine.

At least we tried. :)

Motiejus