Open
Conversation
================================= Enable FastLanes to compile with both Clang (Linux/MacOS) and MSVC (Windows). (This is a first step towards vcpkg packaging, which will make including FastLanes in a project easier. We need Windows not only because it is such a modern and nice OS, but also because it is a native platform for DuckDB and we want to use the future vcpkg version in the a DuckDB extension for FastLanes.) in detail: - New compiler.hpp portability header centralizing compiler-specific macros (vectorize pragmas, __builtin_clz/ctz, diagnostic push/pop) - CMakeLists.txt: accept MSVC alongside Clang, with appropriate flag mappings - Replace ~700 #pragma clang loop vectorize(enable) with FLS_PRAGMA_VECTORIZE - Replace #pragma GCC/clang diagnostic with portable FLS_DIAG_* macros - Add Windows implementation for memory_usage.cpp (GetProcessMemoryInfo) - Add MSVC compat shims to fsst12.h (matching existing fsst.h pattern) - Guard Clang-only -Wno-macro-redefined flags in 8 subdirectory CMakeLists.txt - Add /bigobj, /MP, /Zc:__cplusplus flags for MSVC builds - Split interpreter.cpp into encoding/decoding files to reduce MSVC compile time - Fix C++20 structured binding in range-for (unsupported by MSVC)
…msvc++ compile time passable for windows CI) make format
- add newer macos 26, deprecate 13 - remove debug/true builds as they are slow and flaky - add windows-arm as a platform - add mvsc test and example
…dows On Windows/MSVC, `unsigned long` is 32 bits (vs 64 bits on Linux/macOS). This caused three distinct failures in the test suite: 1. std::stol/stoul throw out_of_range for values that fit in 64 bits but exceed 32-bit range. Replace with std::stoll/stoull in attribute.cpp and rowgroup.cpp. 2. String decode operators reserved 1024 * 2MB = 2GB in a single vector::reserve() call. Linux overcommits virtual memory so this succeeds; Windows does not, throwing bad_alloc. Replace with amortized doubling growth strategy across all 8 decode operators. 3. FSST12 decompressor cast the symbol table to `unsigned long*`, giving 4-byte stride on Windows instead of the required 8-byte stride. This caused symbol lookups at wrong offsets, producing correct-length but wrong-content strings. Fix by casting to `unsigned long long*` in fsst12.h. Also removes debug instrumentation from the reader/materializer that was added while investigating these issues.
…SEH guard-page clash 1. GALP null pointer dereference (all 12 galp tests): make_dec_galp_expr() accessed operand_tokens() which is null for GALP/ALP columns — the encoding path never emits operand tokens. Removed the unnecessary state.cur_operand assignment since dec_alp_opr uses hardcoded segment indices. 2. Iterator invalidation in fill_in() (src/table/rowgroup.cpp): The FLSStrColumn padding loop called push_back() on byte_arr while indexing into the same vector, which is UB when push_back triggers reallocation. Fixed by copying the last value into a separate vector before the loop. 3. SINGLE_COLUMN_JPEG spurious SEH failure (MSVC /MT only): Large heap allocations (~80MB of JPEG base64 data) touch Windows heap-internal guard pages, raising transient 0xc0000005 exceptions. GoogleTest's __try/__except catches these before the heap manager can handle them. Added a VEH handler (test/src/msvc_heap_guard.cpp) that commits the faulting page and resumes execution. Linked as an OBJECT library into all test executables.
(shared linking working) properly with FLS_API
…ts on MSVC The previous FLS_BUILD_DLL/FLS_STATIC definitions were set on the FastLanes target only, but object libraries compile before that target exists and never received the definition. Move to directory-scoped add_compile_definitions in src/CMakeLists.txt so all object libs under src/ get FLS_BUILD_DLL while test targets (under test/) correctly see FLS_API as __declspec(dllimport). Fix MSVC's eager special-member instantiation for dllexport classes: - Delete copy ops on 13 classes with non-copyable members (unique_ptr, variants) - Add out-of-line destructors/move-ctors for classes with incomplete-type unique_ptr members (Segment, RowgroupView, TableView, Reader, RowgroupReader, TableReader, Table) - Include complete type headers where forward declarations are insufficient for dllexport (buf.hpp, column_view.hpp, rowgroup_view.hpp, rowgroup.hpp, table_descriptor.hpp) Add FLS_API to public symbols consumed by tests across the DLL boundary: ValidityMask, Double, Patch, parse_integer (+ explicit instantiations), parse_timestamp, timestamp_formatter, make_decimal, make_decimal_t, sampling_layout_dynamic, is_1_to_1, token_to_string, to_json/from_json, JSON class. Also fix: missing #include <cstddef> for ptrdiff_t in rowgroup.cpp, and duplicate include guard in table_view.hpp (was FLS_READER_ROWGROUP_VIEW_HPP). All 260 tests pass in shared linking mode on Windows ARM64 MSVC. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
peterboncz
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enable FastLanes to compile with both Clang (Linux/MacOS) and MSVC (Windows).
in detail:
(This is a first step towards vcpkg packaging, which will make including FastLanes
in a project easier. We need Windows not only because it is such a modern and nice
OS, but also because it is a native platform for DuckDB and we want to use the
future vcpkg version in the a DuckDB extension for FastLanes.)