[VL] Isolate /usr/local from native builds on macOS#12105
Open
jackylee-ch wants to merge 1 commit into
Open
Conversation
… macOS On Homebrew-based macOS, `/usr/local` accumulates leftover headers, libraries, and CMake packages from prior Velox/Gluten builds (often root-owned after past `sudo make install` runs from `cmake_install` in the bootstrap script). When the native build includes `/usr/local/include` and `/usr/local/lib/cmake` in its CMake search paths it picks up stale or mismatched versions of re2, glog, fmt, etc., shadowing Velox's `deps-install` copies and producing confusing link or compile errors. Concrete failure modes seen: - Fresh build picks up `/usr/local/lib/cmake/glog` from a prior `cmake --install` and links a stale glog into libvelox.dylib alongside the bundled glog from `deps-install`. - `/usr/local/include` shadows Cellar headers (e.g. fmt) and yields `static_assert` failures because Velox's bundled fmt advanced past what `/usr/local/include/fmt` provides. - Link errors against an old re2 left over from an aborted Velox bootstrap. Force CMake to ignore `/usr/local` when building native deps on macOS in three places that drive third-party / Velox / Gluten CMake configs: - `dev/build-helper-functions.sh::cmake_install`: adds `-DCMAKE_NO_SYSTEM_FROM_IMPORTED=ON`, `-DCMAKE_IGNORE_PREFIX_PATH=/usr/local`, `-DCMAKE_IGNORE_PATH`, `-DCMAKE_SYSTEM_IGNORE_PATH` for the third-party CMake invocations driven by the Arrow/Velox bootstrap. - `ep/build-velox/src/build-velox.sh::compile`: same flags when configuring Velox itself, plus suppress two Apple-Clang-only warnings that fire under `-Werror` on macOS but are benign noise: `-Wno-inconsistent-missing-override` and `-Wno-macro-redefined`. - `dev/builddeps-veloxbe.sh::build_gluten_cpp`: same isolation flags plus the same warning suppressions for the Gluten CMake configure. All blocks are gated on `[[ "$(uname)" == "Darwin" ]]` / `if [ $OS == 'Darwin' ]`. Linux paths are unchanged. Without this isolation, the macOS native build is non-deterministic depending on what's left over in `/usr/local`; with it the build only reads from the well-defined Homebrew Cellar prefix (`/opt/homebrew`) and Velox's `deps-install`. Verification: - macOS 14 arm64 with Apple Clang 17 and a `/usr/local` populated with stale glog/re2/fmt installs from a prior session: with these flags applied, the build no longer references `/usr/local/include` or `/usr/local/lib` in any compile or link command (verified via `_build/release/build.ninja`), and produces clean libvelox.dylib + libgluten.dylib that pass the gluten cpp ctest matrix. - Linux x86_64 build green; all blocks are Darwin-gated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
On Homebrew-based macOS,
/usr/localaccumulates leftover headers,libraries, and CMake packages from prior Velox/Gluten builds (often
root-owned after past
sudo make installruns fromcmake_installin the bootstrap). When the native build includes
/usr/local/includeand
/usr/local/lib/cmakein its CMake search paths it picks up staleor mismatched versions of re2, glog, fmt, etc., shadowing Velox's
deps-installcopies and producing confusing link or compile errors.Concrete failure modes that show up before this fix:
/usr/local/lib/cmake/glogfrom a priorcmake --installand links a stale glog intolibvelox.dylibalongside the bundled glog from
deps-install./usr/local/includeshadows Cellar headers (e.g. fmt) and yieldsstatic_assertfailures because Velox's bundled fmt advanced pastwhat
/usr/local/include/fmtprovides.bootstrap.
This PR forces CMake to ignore
/usr/localwhen building native depson macOS, in three places that drive the third-party / Velox / Gluten
CMake configures:
dev/build-helper-functions.sh::cmake_install-DCMAKE_NO_SYSTEM_FROM_IMPORTED=ON,-DCMAKE_IGNORE_PREFIX_PATH=/usr/local,-DCMAKE_IGNORE_PATH,-DCMAKE_SYSTEM_IGNORE_PATHfor the third-party CMake invocations driven by the Arrow/Velox bootstrapep/build-velox/src/build-velox.sh::compile-Werroron macOS but are benign noise:-Wno-inconsistent-missing-overrideand-Wno-macro-redefineddev/builddeps-veloxbe.sh::build_gluten_cppAll blocks are gated on
[[ "$(uname)" == "Darwin" ]]/if [ $OS == 'Darwin' ]. Linux build/link semantics are unchanged.Without this isolation, the macOS native build is non-deterministic
depending on what's left over in
/usr/local; with it, the build onlyreads from the well-defined Homebrew Cellar prefix (
/opt/homebrew)and Velox's
deps-install.How was this patch tested?
/usr/localpopulatedwith stale glog/re2/fmt installs from a prior session: with these
flags applied, the build no longer references
/usr/local/includeor
/usr/local/libin any compile or link command (verified via_build/release/build.ninja), and produces cleanlibvelox.dyliblibgluten.dylibthat pass the Gluten CPP ctest matrix.(VeloxBloomFilterTest, ColumnarBatchTest, VeloxListenerApiTest,
OnHeapFileSystemTest, ArrowColumnVectorTest) all pass on the
resulting build.
Darwin-gated. Local Ubuntu build verified clean.
Was this patch authored or co-authored using generative AI tooling?
co-auth: Claude (Sonnet/Opus) via Claude Code 1.x