build: Only replace `cl.exe` with `clang-cl` for ARM64 Windows builds #2216

MarijnS95 · 2025-01-10T13:44:02Z

Caution

This change depends on rust-lang/cc-rs#1357.

When cross-compiling to Windows from Linux or similar, it's common to use the clang-cl driver from the LLVM toolchain which supports parsing cl.exe-like arguments.

Ring however doesn't seem to compile for ARM64 Windows using cl.exe, and contains a // FIXME-style workaround to use clang to compile its C files instead.

The command-line interface for clang isn't always compatible with that of cl.exe and clang-cl. There didn't seem to be any trouble with this yet, but when cross-compiling from Linux it's common to explicitly provide "sysroots" via -vctoolsdir and -winsdkdir in CFLAGS. In such a setup this workaround in ring would pass those arguments to clang, resulting in "unknown argument" errors.

cc-rs can tell us exactly what compiler it found, and we can use this information to decide how to fill in this workaround. If the user was already compiling their code with clang-cl, nothing has to be replaced. In the end, all this entails is changing the workaround to not compile with clang, but with clang-cl instead.

briansmith

In ci.yml, we have (twice):

      - if: ${{ matrix.target == 'aarch64-pc-windows-msvc' }}
        run: |
          echo "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\Llvm\x64\bin" >> $GITHUB_PATH

When using clang-cl, is this still necessary, or is clang-cl already in the path in GitHub actions?

If clang-cl is more common and clang-cl is already in the path, then we should remove this from ci.yml.

We should also update BUILDING.md "For Windows ARM64 targets..." to describe the updated situation.

build.rs

briansmith · 2025-01-10T18:39:20Z

build.rs

+        && !compiler.is_like_clang()
+        && !compiler.is_like_clang_cl()
+    {
+        c.compiler("clang");


Should we start defaulting to clang-cl instead?

Perhaps. If users are more commonly compiling for Windows MSVC than Windows GNU, arbitrarily replacing their requested compiler with clang-cl whose arguments are compatible with cl.exe seems a lot less error-prone than giving them the clang interface. Any cl.exe-specific argument in their CFLAGS (my issue: #2215) and this fails to compile.

Either way, overwriting compilers - which might have been carefully set up via CC environment flags etc (to not have to rely on PATH!) - seems very painful. As said below, having ring compile natively with cl.exe would be awesome. At least it's isolated to ARM64 Windows though.

At the time, cl.exe didn't have uint128_t and probably there were other issues too. Maybe BoringSSL actually made some changes upstream to avoid requiring uint128_t for the ECC code? If so, then we can probably start supporting cl.exe. Definitely, I would love it, and I think there's already an issue for it.

Cool - I'll try to cross-compile with cl.exe from my Windows X64 machine to find out and document what specifically is holding it back if at all.

At the same time I'm inquiring to get access to an ARM64 Windows machine soon, to test what happens when natively compiling on that machine.

We would point out that uint128_t is not standardised at all either in C++ or C so sin-binning the compiler for not supporting something that isn't even a part of the language standard seems misguided as this indicates broken code needing fixed.

briansmith · 2025-01-10T18:40:15Z

In the commit message:

build: Don't overwrite clang-cl with clang for ARM64 Windows builds

here and elsewhere you use the word "overwrite" when I think "override" is the better word.

When ring however overwrites this compiler with clang,

I suggest instead we say "When this is done,".

existing user arguments
in i.e. CFLAGS are not compatible resulting in various "unknown
argument" errors such as for -vctoolsdir and -winsdkdir.

Most likely, we need to modify our compiler flag handling in build.rs to pass the correct flags to clang-cl.

As pointed out in the original ARM64 Windows support PR, compiling
ring with clang-cl works equally well as clang: allow that by not
overwriting the compiler with clang if it's already clang-cl, to not
have to sort out incompatible arguments.

IIRC, when I tried it, clang-cl couldn't compile the assembly source files, which is why we hard-code clang.

We need to modify ci.yml to add the clang-cl-based configuration.

I also think that, assuming it works "equally well," we should default to the clang-cl-based build (instead of clang) when we detect the C compiler is MSVC and the target is aarch64.

MarijnS95 · 2025-01-10T18:57:31Z

In ci.yml, we have (twice):
      - if: ${{ matrix.target == 'aarch64-pc-windows-msvc' }}
        run: |
          echo "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\Llvm\x64\bin" >> $GITHUB_PATH
When using clang-cl, is this still necessary, or is clang-cl already in the path in GitHub actions?

If clang-cl is more common and clang-cl is already in the path, then we should remove this from ci.yml.

This PR is solely about supporting a cross-compile from Linux (for #2215), and I have no clue about a "native" cross-compile from Windows->Windows (barring the architecture). If you want to know this for sure, I'd have to boot into a Windows machine for some experimentation.

It seems very unlikely that clang-cl is available on PATH any more than cl.exe; it is a specific tool to get a cl.exe-like command line interface while using the LLVM compiler stack.

We should also update BUILDING.md "For Windows ARM64 targets..." to describe the updated situation.

I don't think much is changing, or supposed to be changed here. All this PR really is about, is making the constraints for this // FIXME workaround more strict so that it doesn't replace the requested/provided compiler too eagerly unless absolutely deemed necessary.

As above, if we're "blindly" replacing the compiler command with "clang" without ever looking at the compiler-specific arguments and whether they're compatible, we better isolate such workarounds thoroughly.

The best course of action would be if this project could natively compile for ARM64 with cl.exe so that none of this "compiler-command replacing" is necessary, though I didn't read up on how feasible that would be.

MarijnS95 · 2025-01-10T19:26:18Z

In the commit message:

build: Don't overwrite clang-cl with clang for ARM64 Windows builds

here and elsewhere you use the word "overwrite" when I think "override" is the better word.

Agreed. It doesn't overwrite the compiler, but it does "overwrite" or "overrule" a CC environment flag or similar in the users' environment.

When ring however overwrites this compiler with clang,

I suggest instead we say "When this is done,".

NAK. I didn't introduce the idea of "overwrite the compiler with clang" yet before this part of the paragraph, so this would refer to nothing.

existing user arguments
in i.e. CFLAGS are not compatible resulting in various "unknown
argument" errors such as for -vctoolsdir and -winsdkdir.

Most likely, we need to modify our compiler flag handling in build.rs to pass the correct flags to clang-cl.

As above, if we assume that user flags are more likely compatible with cl.exe than with clang, using clang-cl is the better option than picking clang.

I do think we can be more strict here. We're going to expose the .family() flag in cc-rs, so I'm thinking this:

match compiler.family() {
    // The workaround
    ToolFamily::Msvc { clang_cl: false } => c.compiler("clang-cl"),
    ToolFamily::Msvc { clang_cl: true } | ToolFamily::Clang { .. } => {},
    ToolFamily::Gnu { .. } => todo!("Test this"),
}

As pointed out in the original ARM64 Windows support PR, compiling
ring with clang-cl works equally well as clang: allow that by not
overwriting the compiler with clang if it's already clang-cl, to not
have to sort out incompatible arguments.

IIRC, when I tried it, clang-cl couldn't compile the assembly source files, which is why we hard-code clang.

Curious what that entails. For me it failed only because of warnings forced as errors in the git tree:

ring/build.rs

Line 314 in d2e401f

let force_warnings_into_errors = is_git;

We need to modify ci.yml to add the clang-cl-based configuration.

I also think that, assuming it works "equally well," we should default to the clang-cl-based build (instead of clang) when we detect the C compiler is MSVC and the target is aarch64.

Sure let's always use clang-cl.

briansmith · 2025-01-10T19:46:08Z

Curious what that entails. For me it failed only because of warnings forced as errors in the git tree:

In the issue, you mentioned:

error: mixing declarations and code is incompatible with standards before C99 [-Werror,-Wdeclaration-after-statement]

Of course, we require C99 at least, so we need to change build.rs to tell clang-cl to use the right version of C. You can see here we have:

        static NON_MSVC_FLAGS: &[&str] = &[
            "-fvisibility=hidden",
            "-std=c1x", // GCC 4.6 requires "c1x" instead of "c11"
            "-Wall",
            "-Wbad-function-cast",
            "-Wcast-align",
            "-Wcast-qual",
            "-Wconversion",
            "-Wmissing-field-initializers",
            "-Wmissing-include-dirs",
            "-Wnested-externs",
            "-Wredundant-decls",
            "-Wshadow",
            "-Wsign-compare",
            "-Wsign-conversion",
            "-Wstrict-prototypes",
            "-Wundef",
            "-Wuninitialized",
        ];

So, when the compiler is clang-cl, we need to pass the clang-cl equivalent of -std=c11 .

In the issue, you also mentioned:

error: unsafe buffer access [-Werror,-Wunsafe-buffer-usage]

Obviously, this is pretty concerning and I'd like to prioritize addressing it.

The reason we have all of these warnings enabled in the .git build is precisely to ensure compatibility with the compilers we support.

If we're adding support for a new compiler, as we are here with clang-cl, then we need to have CI test this configuration. Which means doing the ci.yml changes and getting the build to pass in CI in the .git configuration where all the warnings are enabled.

briansmith · 2025-01-10T19:46:54Z

NAK. I didn't introduce the idea of "overwrite the compiler with clang" yet before this part of the paragraph, so this would refer to nothing.

It is introduced by the subject line "Don't override clang-cl with clang for ARM64 Windows builds."

briansmith · 2025-01-10T20:03:21Z

BUILDING.md

+Add Microsoft's provided version of `clang-cl` to `%PATH%`, which will allow the
 build to work in GitHub Actions without installing anything:
 ```
 $env:Path += ";C:\Program Files (x86)\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\x64\bin"


Is this where clang-cl lives?

It's part of the LLVM toolchain. On Linux machines, yes. On Windows: probably?

BUILDING.md

briansmith · 2025-01-10T20:05:07Z

build.rs

-    let compiler = if target.os == WINDOWS && target.arch == AARCH64 && !compiler.is_like_clang() {
-        let _ = c.compiler("clang");
-        c.get_compiler()
+    // FIXME: On Windows AArch64 we currently must use LLVM (clang-cl) to compile C code


I think we should say "clang or clang-cl" instead of "clang-cl" here. I think we should add a comment "If the compiler is MSVC, switch to clang-cl so that CFLAGS, etc., are more likely to be compatible."

I had that at first, but considered that the weight is mostly on the file being compiled via an LLVM toolchain. Whether the compiler/argument "driver" is clang or clang-cl doesn't matter, but you're right that I could comment this better in the match arms.

MarijnS95 · 2025-01-10T20:06:25Z

I've rewritten the patch and description:

We now use clang-cl in place of clang, because:
1. The error I got when cross-compiling (passing cl.exe arguments to clang instead of clang-cl);
2. If the user is already compiling with clang, nothing needs to be changed;
3. Compiling with GNU is currently not supported (but is a valid target);
This means the clang-cl path is now natively tested in CI.

Note that this means we have the following targets to add to CI:

(Cross-)compiling from a Windows machine to ARM64 Windows using the default cl.exe (triggers the // FIXME to replace it with clang-cl);
(Cross-)compiling from a Windows machine to ARM64 Windows using LLVM / clang;
Cross-compiling from a Linux machine to ARM64 Windows with LLVM (my local target for testing);
❔ Compiling natively on an ARM64 Windows machine using the default cl.exe;
❔ Compiling natively on an ARM64 Windows machine using LLVM / clang.

build.rs

codecov · 2025-01-10T20:26:57Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.16%. Comparing base (d2e401f) to head (ac6451a).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2216      +/-   ##
==========================================
+ Coverage   97.02%   97.16%   +0.13%     
==========================================
  Files         160      158       -2     
  Lines       20391    20330      -61     
  Branches      458      455       -3     
==========================================
- Hits        19785    19754      -31     
+ Misses        498      471      -27     
+ Partials      108      105       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

MarijnS95 · 2025-01-10T20:48:25Z

In ci.yml, we have (twice):
      - if: ${{ matrix.target == 'aarch64-pc-windows-msvc' }}
        run: |
          echo "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\Llvm\x64\bin" >> $GITHUB_PATH
When using clang-cl, is this still necessary, or is clang-cl already in the path in GitHub actions?

Going all the way back to this, I'm confused by the CI setup:

For the test: job, there's a massive target: list. However, no host_os is set (mandatory for runs-on), so I assume only the explicit pairs in include: are actually running and this list is redundant?

For the "twice" case you mentioned, in test-features: there's no aarch64-pc-windows-msvc present in the matrix at all so if: ${{ matrix.target == 'aarch64-pc-windows-msvc' }} will likely never hit?

MarijnS95 · 2025-03-14T15:37:54Z

Regarding availability of clang-cl again, I was experimenting with our Debian bookworm runner image (Linux), and it doesn't provide clang-cl from neither the llvm-15 nor the clang-15 package, you need to install clang-tools-15 to get access to that.

Before however, our clang-cl was a simple symlink to /usr/lib/llvm-15/bin/clang (and llvm-bin was a symlink to llvm-ar, since these binaries likely look at argv[0] to understand in what mode they're supposed to be running). Not sure why it's a binary blob in that package, maybe it's a copy of clang for no reason.

Jasper-Bekkers · 2025-09-26T11:59:11Z

@briansmith At this stage this PR is a hand full of buildflag changes only and we've been running with it (and cross compiling with this PR) for about 6 months ourselves already. Would it be possible to get this merged into mainline or are there still significant objections to the content of the PR?

dragonmux

Your changes for which warnings to disable goes far too far - silencing all warnings being produced indiscriminately is not the way. The two extra warnings this PR now silences should really be errors from a Rust perspective because they are the compiler telling you about legitimate bugs. Using reserved identifiers the compiler may itself be using, accessing buffers in a way that it is finding via static analysis likely leads to a buffer overflow (this is what unsafe buffer usage implies).

build.rs

When cross-compiling to Windows from Linux or similar, it's common to use the `clang-cl` driver from the LLVM toolchain which supports parsing `cl.exe`-like arguments. Ring however doesn't seem to compile for ARM64 Windows using `cl.exe`, and contains a `// FIXME`-style workaround to use `clang` to compile its C files instead. The command-line interface for `clang` isn't always compatible with that of `cl.exe` and `clang-cl`. There didn't seem to be any trouble with this yet, but when cross-compiling from Linux it's common to explicitly provide "sysroots" via `-vctoolsdir` and `-winsdkdir` in `CFLAGS`. In such a setup this workaround in `ring` would pass those arguments to `clang`, resulting in "unknown argument" errors. `cc-rs` can tell us exactly what compiler it found, and we can use this information to decide how to fill in this workaround. If the user was already compiling their code with `clang-cl`, nothing has to be replaced. In the end, all this entails is changing the workaround to not compile with `clang`, but with `clang-cl` instead.

`is_msvc_not_clang_cl` is already defined above through a recent change.

`/Wall` is treated differently on `clang-cl` versus `cl.exe`, requiring different warnings to be configured.

oskirby · 2025-12-08T20:16:48Z

We would love to see this merged into mainline as well in order to unblock some of our builds on the aarch64-pc-windows-msvc target.

MarijnS95 · 2025-12-09T11:12:08Z

@briansmith is there anything we can do to further this PR? I could rebase it if it helps, but there are no conflicts at the moment.

briansmith requested changes Jan 10, 2025

View reviewed changes

build.rs Outdated Show resolved Hide resolved

briansmith reviewed Jan 10, 2025

View reviewed changes

MarijnS95 changed the title ~~build: Don't overwrite clang-cl with clang for ARM64 Windows builds~~ build: Don't override clang-cl with clang for ARM64 Windows builds Jan 10, 2025

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch 2 times, most recently from 59f3144 to 9608023 Compare January 10, 2025 19:59

This comment was marked as resolved.

Sign in to view

MarijnS95 changed the title ~~build: Don't override clang-cl with clang for ARM64 Windows builds~~ build: Only replace cl.exe with clang-cl for ARM64 Windows builds Jan 10, 2025

briansmith reviewed Jan 10, 2025

View reviewed changes

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from 9608023 to ac6451a Compare January 10, 2025 20:13

briansmith reviewed Jan 10, 2025

View reviewed changes

build.rs Outdated Show resolved Hide resolved

build.rs Outdated Show resolved Hide resolved

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from ac6451a to 3fac180 Compare January 10, 2025 20:43

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from 3fac180 to d44f9d1 Compare January 10, 2025 20:57

MarijnS95 mentioned this pull request Jan 12, 2025

Add "prefer to use clang instead of MSVC" API that can automatically find Microsoft-provided clang rust-lang/cc-rs#882

Closed

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from d44f9d1 to ce7d500 Compare March 12, 2025 09:00

This comment was marked as resolved.

Sign in to view

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from ce7d500 to 80f1285 Compare March 12, 2025 09:52

shadaj mentioned this pull request Mar 29, 2025

ci: fix python hydrocli build hydro-project/hydro#1818

Open

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch 3 times, most recently from 0c55005 to b16e53e Compare March 31, 2025 12:05

MarijnS95 mentioned this pull request Aug 4, 2025

Conflicting C(XX)FLAGS when cross-compiling to aarch64-pc-windows-msvc #2215

Open

MarijnS95 mentioned this pull request Aug 4, 2025

Windows arm64: enable building with MSVC #2673

Closed

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from 612555f to f3ac558 Compare August 5, 2025 10:13

dennisameling mentioned this pull request Sep 6, 2025

Windows arm64: use cc-rs' prefer_clang_cl_over_msvc #2699

Open

MarijnS95 requested a review from briansmith September 26, 2025 12:00

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch 2 times, most recently from a92005a to e08508e Compare October 15, 2025 13:02

This comment was marked as outdated.

Sign in to view

dragonmux suggested changes Oct 16, 2025

View reviewed changes

build.rs Outdated Show resolved Hide resolved

build.rs Outdated Show resolved Hide resolved

MarijnS95 added 3 commits October 16, 2025 17:50

build: Factor out a common variable

3b13d6d

`is_msvc_not_clang_cl` is already defined above through a recent change.

build: Use separate warnings for clang-cl versus cl.exe

2cc722a

`/Wall` is treated differently on `clang-cl` versus `cl.exe`, requiring different warnings to be configured.

MarijnS95 force-pushed the aarch64-windows-allow-clang-cl branch from e08508e to 2cc722a Compare October 16, 2025 16:11

oskirby mentioned this pull request Dec 8, 2025

VPN-2199: Windows ARM Support mozilla-mobile/mozilla-vpn-client#10891

Merged

5 tasks

cyqsimon mentioned this pull request Dec 10, 2025

fix: clang-cl libpath problems rust-cross/cargo-xwin#173

Merged

build: Only replace cl.exe with clang-cl for ARM64 Windows builds #2216

Are you sure you want to change the base?

build: Only replace cl.exe with clang-cl for ARM64 Windows builds #2216

Conversation

MarijnS95 commented Jan 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

briansmith left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MarijnS95 Jan 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

briansmith commented Jan 10, 2025

Uh oh!

MarijnS95 commented Jan 10, 2025

Uh oh!

MarijnS95 commented Jan 10, 2025

Uh oh!

briansmith commented Jan 10, 2025

Uh oh!

briansmith commented Jan 10, 2025

Uh oh!

This comment was marked as resolved.

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MarijnS95 commented Jan 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jan 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

MarijnS95 commented Jan 10, 2025

Uh oh!

This comment was marked as resolved.

MarijnS95 commented Mar 14, 2025

Uh oh!

Jasper-Bekkers commented Sep 26, 2025

Uh oh!

This comment was marked as outdated.

dragonmux left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

oskirby commented Dec 8, 2025

Uh oh!

MarijnS95 commented Dec 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

build: Only replace `cl.exe` with `clang-cl` for ARM64 Windows builds #2216

build: Only replace `cl.exe` with `clang-cl` for ARM64 Windows builds #2216

MarijnS95 commented Jan 10, 2025 •

edited

Loading

MarijnS95 Jan 10, 2025 •

edited

Loading

MarijnS95 commented Jan 10, 2025 •

edited

Loading

codecov bot commented Jan 10, 2025 •

edited

Loading

dragonmux left a comment •

edited

Loading