WIP: Upgrade DataFusion to arrow-rs/parquet 58.0.0 #19728

alamb · 2026-01-10T12:19:14Z

Which issue does this PR close?

Follow on to WIP: Upgrade DataFusion to arrow-rs/parquet 57.2.0 #19355
related to Release arrow-rs / parquet Major version 58.0.0 (January 2026) arrow-rs#8466

Rationale for this change

Keep datafusion up to date (and test Arrow using DataFusion tests)

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

….2.0

alamb · 2026-01-10T12:30:27Z

datafusion/sqllogictest/test_files/spark/datetime/make_interval.slt

 # We expect something like 0.00 secs with query ?
 query T
 SELECT make_interval(0, 0, 0, 0, 0, 0, 0.0) || '';
 ----


This is due to @Jefffrey 's improvement in

fix: display 0 secs for empty DayTime/MonthDayNano intervals arrow-rs#9023

close #17455

Probably should remove the accompanying comments too, these

# Intervals being rendered as empty string, see issue: # https://github.com/apache/datafusion/issues/17455 # We expect something like 0.00 secs with query ?

alamb · 2026-01-10T13:42:26Z

datafusion-cli/src/main.rs

        | alltypes_plain.parquet            | 1851            | 8882                | 2    | page_index=false |
-        | alltypes_tiny_pages.parquet       | 454233          | 269266              | 2    | page_index=true  |
-        | lz4_raw_compressed_larger.parquet | 380836          | 1347                | 2    | page_index=false |
+        | alltypes_tiny_pages.parquet       | 454233          | 269074              | 2    | page_index=true  |


I think this reduction in metadata size is a direct consequence of @WaterWhisperer's PR to improve PageEncoding representation

feat: change default behavior for Parquet PageEncodingStats to bitmask arrow-rs#9051

Dandandan · 2026-01-10T14:18:49Z

Run benchmarks

alamb-ghbot · 2026-01-10T17:46:31Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_58 (35b97fa) to b9a3b9f diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-01-10T18:26:55Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_58
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query    ┃        HEAD ┃ alamb_update_arrow_58 ┃    Change ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0 │  2354.54 ms │            2406.77 ms │ no change │
│ QQuery 1 │   939.82 ms │             929.76 ms │ no change │
│ QQuery 2 │  1882.93 ms │            1891.80 ms │ no change │
│ QQuery 3 │  1163.47 ms │            1165.13 ms │ no change │
│ QQuery 4 │  2306.07 ms │            2241.55 ms │ no change │
│ QQuery 5 │ 28452.06 ms │           28080.09 ms │ no change │
│ QQuery 6 │  4018.71 ms │            4055.12 ms │ no change │
│ QQuery 7 │  3790.29 ms │            3670.14 ms │ no change │
└──────────┴─────────────┴───────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 44907.89ms │
│ Total Time (alamb_update_arrow_58)   │ 44440.36ms │
│ Average Time (HEAD)                  │  5613.49ms │
│ Average Time (alamb_update_arrow_58) │  5555.05ms │
│ Queries Faster                       │          0 │
│ Queries Slower                       │          0 │
│ Queries with No Change               │          8 │
│ Queries with Failure                 │          0 │
└──────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ alamb_update_arrow_58 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     1.86 ms │               1.94 ms │     no change │
│ QQuery 1  │    50.56 ms │              50.42 ms │     no change │
│ QQuery 2  │   134.60 ms │             134.00 ms │     no change │
│ QQuery 3  │   156.94 ms │             152.15 ms │     no change │
│ QQuery 4  │  1074.92 ms │            1227.05 ms │  1.14x slower │
│ QQuery 5  │  1349.11 ms │            1512.35 ms │  1.12x slower │
│ QQuery 6  │     1.82 ms │               1.86 ms │     no change │
│ QQuery 7  │    56.21 ms │              53.80 ms │     no change │
│ QQuery 8  │  1404.44 ms │            1546.63 ms │  1.10x slower │
│ QQuery 9  │  1744.32 ms │            1884.58 ms │  1.08x slower │
│ QQuery 10 │   344.43 ms │             351.00 ms │     no change │
│ QQuery 11 │   393.88 ms │             399.97 ms │     no change │
│ QQuery 12 │  1259.31 ms │            1442.13 ms │  1.15x slower │
│ QQuery 13 │  1916.02 ms │            2053.79 ms │  1.07x slower │
│ QQuery 14 │  1228.80 ms │            1371.03 ms │  1.12x slower │
│ QQuery 15 │  1223.04 ms │            1359.64 ms │  1.11x slower │
│ QQuery 16 │  2568.94 ms │            2675.51 ms │     no change │
│ QQuery 17 │  2521.47 ms │            2647.03 ms │     no change │
│ QQuery 18 │  6120.19 ms │            5000.68 ms │ +1.22x faster │
│ QQuery 19 │   120.13 ms │             119.44 ms │     no change │
│ QQuery 20 │  1948.33 ms │            1896.06 ms │     no change │
│ QQuery 21 │  2245.14 ms │            2178.01 ms │     no change │
│ QQuery 22 │  3847.87 ms │            3802.23 ms │     no change │
│ QQuery 23 │ 21572.08 ms │           12373.92 ms │ +1.74x faster │
│ QQuery 24 │   225.72 ms │             216.64 ms │     no change │
│ QQuery 25 │   471.41 ms │             478.99 ms │     no change │
│ QQuery 26 │   218.83 ms │             235.13 ms │  1.07x slower │
│ QQuery 27 │  2823.98 ms │            2691.15 ms │     no change │
│ QQuery 28 │ 23860.88 ms │           23263.75 ms │     no change │
│ QQuery 29 │   982.83 ms │             966.29 ms │     no change │
│ QQuery 30 │  1343.19 ms │            1378.01 ms │     no change │
│ QQuery 31 │  1521.78 ms │            1379.20 ms │ +1.10x faster │
│ QQuery 32 │  5104.40 ms │            4890.24 ms │     no change │
│ QQuery 33 │  5837.64 ms │            5506.92 ms │ +1.06x faster │
│ QQuery 34 │  5933.39 ms │            5605.95 ms │ +1.06x faster │
│ QQuery 35 │  1921.56 ms │            1960.63 ms │     no change │
│ QQuery 36 │    65.96 ms │              68.18 ms │     no change │
│ QQuery 37 │    45.85 ms │              45.16 ms │     no change │
│ QQuery 38 │    66.53 ms │              65.89 ms │     no change │
│ QQuery 39 │   102.71 ms │             106.18 ms │     no change │
│ QQuery 40 │    28.23 ms │              25.52 ms │ +1.11x faster │
│ QQuery 41 │    24.10 ms │              23.51 ms │     no change │
│ QQuery 42 │    19.09 ms │              18.80 ms │     no change │
└───────────┴─────────────┴───────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 103882.45ms │
│ Total Time (alamb_update_arrow_58)   │  93161.37ms │
│ Average Time (HEAD)                  │   2415.87ms │
│ Average Time (alamb_update_arrow_58) │   2166.54ms │
│ Queries Faster                       │           6 │
│ Queries Slower                       │           9 │
│ Queries with No Change               │          28 │
│ Queries with Failure                 │           0 │
└──────────────────────────────────────┴─────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ alamb_update_arrow_58 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 119.70 ms │             118.08 ms │     no change │
│ QQuery 2  │  35.82 ms │              36.60 ms │     no change │
│ QQuery 3  │  40.70 ms │              36.99 ms │ +1.10x faster │
│ QQuery 4  │  30.37 ms │              30.72 ms │     no change │
│ QQuery 5  │  91.59 ms │              90.47 ms │     no change │
│ QQuery 6  │  21.09 ms │              20.76 ms │     no change │
│ QQuery 7  │ 222.34 ms │             225.43 ms │     no change │
│ QQuery 8  │  44.85 ms │              40.29 ms │ +1.11x faster │
│ QQuery 9  │ 122.80 ms │             104.76 ms │ +1.17x faster │
│ QQuery 10 │  81.06 ms │              66.33 ms │ +1.22x faster │
│ QQuery 11 │  23.80 ms │              24.89 ms │     no change │
│ QQuery 12 │  52.22 ms │              52.78 ms │     no change │
│ QQuery 13 │  48.94 ms │              49.75 ms │     no change │
│ QQuery 14 │  14.95 ms │              14.96 ms │     no change │
│ QQuery 15 │  30.50 ms │              30.48 ms │     no change │
│ QQuery 16 │  28.47 ms │              29.06 ms │     no change │
│ QQuery 17 │ 151.61 ms │             154.03 ms │     no change │
│ QQuery 18 │ 287.03 ms │             288.27 ms │     no change │
│ QQuery 19 │  39.57 ms │              39.77 ms │     no change │
│ QQuery 20 │  57.69 ms │              57.36 ms │     no change │
│ QQuery 21 │ 322.81 ms │             318.54 ms │     no change │
│ QQuery 22 │  21.98 ms │              21.61 ms │     no change │
└───────────┴───────────┴───────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 1889.89ms │
│ Total Time (alamb_update_arrow_58)   │ 1851.95ms │
│ Average Time (HEAD)                  │   85.90ms │
│ Average Time (alamb_update_arrow_58) │   84.18ms │
│ Queries Faster                       │         4 │
│ Queries Slower                       │         0 │
│ Queries with No Change               │        18 │
│ Queries with Failure                 │         0 │
└──────────────────────────────────────┴───────────┘

Dandandan · 2026-01-10T18:32:08Z

run benchmarks

Dandandan · 2026-01-10T18:32:18Z

run benchmark tpch

alamb-ghbot · 2026-01-10T18:40:16Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_58 (35b97fa) to b9a3b9f diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-01-10T19:18:22Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_58
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query    ┃        HEAD ┃ alamb_update_arrow_58 ┃    Change ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0 │  2389.61 ms │            2355.66 ms │ no change │
│ QQuery 1 │   975.65 ms │             969.78 ms │ no change │
│ QQuery 2 │  1875.49 ms │            1871.78 ms │ no change │
│ QQuery 3 │  1174.32 ms │            1145.53 ms │ no change │
│ QQuery 4 │  2258.94 ms │            2223.01 ms │ no change │
│ QQuery 5 │ 27824.57 ms │           28406.72 ms │ no change │
│ QQuery 6 │  4031.59 ms │            4002.29 ms │ no change │
│ QQuery 7 │  3463.56 ms │            3533.17 ms │ no change │
└──────────┴─────────────┴───────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 43993.73ms │
│ Total Time (alamb_update_arrow_58)   │ 44507.93ms │
│ Average Time (HEAD)                  │  5499.22ms │
│ Average Time (alamb_update_arrow_58) │  5563.49ms │
│ Queries Faster                       │          0 │
│ Queries Slower                       │          0 │
│ Queries with No Change               │          8 │
│ Queries with Failure                 │          0 │
└──────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ alamb_update_arrow_58 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     1.90 ms │               1.92 ms │     no change │
│ QQuery 1  │    52.12 ms │              50.35 ms │     no change │
│ QQuery 2  │   142.92 ms │             133.95 ms │ +1.07x faster │
│ QQuery 3  │   154.00 ms │             152.77 ms │     no change │
│ QQuery 4  │  1072.51 ms │            1063.15 ms │     no change │
│ QQuery 5  │  1369.60 ms │            1320.07 ms │     no change │
│ QQuery 6  │     1.82 ms │               1.86 ms │     no change │
│ QQuery 7  │    54.39 ms │              52.98 ms │     no change │
│ QQuery 8  │  1446.73 ms │            1417.75 ms │     no change │
│ QQuery 9  │  1835.01 ms │            1706.51 ms │ +1.08x faster │
│ QQuery 10 │   344.52 ms │             354.29 ms │     no change │
│ QQuery 11 │   392.19 ms │             397.62 ms │     no change │
│ QQuery 12 │  1286.14 ms │            1275.41 ms │     no change │
│ QQuery 13 │  1947.40 ms │            1962.96 ms │     no change │
│ QQuery 14 │  1259.13 ms │            1231.11 ms │     no change │
│ QQuery 15 │  1247.90 ms │            1222.46 ms │     no change │
│ QQuery 16 │  2534.25 ms │            2515.84 ms │     no change │
│ QQuery 17 │  2514.25 ms │            2527.22 ms │     no change │
│ QQuery 18 │  5440.35 ms │            4806.73 ms │ +1.13x faster │
│ QQuery 19 │   119.26 ms │             118.76 ms │     no change │
│ QQuery 20 │  1929.28 ms │            1872.84 ms │     no change │
│ QQuery 21 │  2213.86 ms │            2178.92 ms │     no change │
│ QQuery 22 │  4264.32 ms │            3784.45 ms │ +1.13x faster │
│ QQuery 23 │ 20258.80 ms │           12236.28 ms │ +1.66x faster │
│ QQuery 24 │   215.95 ms │             219.20 ms │     no change │
│ QQuery 25 │   488.91 ms │             481.13 ms │     no change │
│ QQuery 26 │   226.35 ms │             216.76 ms │     no change │
│ QQuery 27 │  2799.99 ms │            2713.67 ms │     no change │
│ QQuery 28 │ 23840.67 ms │           23084.98 ms │     no change │
│ QQuery 29 │   974.43 ms │             971.18 ms │     no change │
│ QQuery 30 │  1351.52 ms │            1299.11 ms │     no change │
│ QQuery 31 │  1399.75 ms │            1421.76 ms │     no change │
│ QQuery 32 │  4761.26 ms │            4490.64 ms │ +1.06x faster │
│ QQuery 33 │  5521.96 ms │            5313.98 ms │     no change │
│ QQuery 34 │  5548.81 ms │            5584.56 ms │     no change │
│ QQuery 35 │  1979.74 ms │            1851.73 ms │ +1.07x faster │
│ QQuery 36 │    66.91 ms │              67.24 ms │     no change │
│ QQuery 37 │    46.16 ms │              47.24 ms │     no change │
│ QQuery 38 │    66.08 ms │              67.97 ms │     no change │
│ QQuery 39 │   102.12 ms │             100.78 ms │     no change │
│ QQuery 40 │    28.61 ms │              27.21 ms │     no change │
│ QQuery 41 │    23.45 ms │              22.02 ms │ +1.06x faster │
│ QQuery 42 │    19.24 ms │              19.38 ms │     no change │
└───────────┴─────────────┴───────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 101344.52ms │
│ Total Time (alamb_update_arrow_58)   │  90386.74ms │
│ Average Time (HEAD)                  │   2356.85ms │
│ Average Time (alamb_update_arrow_58) │   2102.02ms │
│ Queries Faster                       │           8 │
│ Queries Slower                       │           0 │
│ Queries with No Change               │          35 │
│ Queries with Failure                 │           0 │
└──────────────────────────────────────┴─────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ alamb_update_arrow_58 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 118.62 ms │             117.02 ms │     no change │
│ QQuery 2  │  37.93 ms │              38.57 ms │     no change │
│ QQuery 3  │  41.97 ms │              37.52 ms │ +1.12x faster │
│ QQuery 4  │  31.39 ms │              30.22 ms │     no change │
│ QQuery 5  │  91.63 ms │              91.98 ms │     no change │
│ QQuery 6  │  21.13 ms │              20.89 ms │     no change │
│ QQuery 7  │ 229.95 ms │             228.06 ms │     no change │
│ QQuery 8  │  39.25 ms │              37.93 ms │     no change │
│ QQuery 9  │ 107.60 ms │             107.66 ms │     no change │
│ QQuery 10 │  69.45 ms │              69.75 ms │     no change │
│ QQuery 11 │  23.16 ms │              24.35 ms │  1.05x slower │
│ QQuery 12 │  53.07 ms │              52.75 ms │     no change │
│ QQuery 13 │  51.17 ms │              48.55 ms │ +1.05x faster │
│ QQuery 14 │  15.28 ms │              15.11 ms │     no change │
│ QQuery 15 │  30.14 ms │              30.73 ms │     no change │
│ QQuery 16 │  31.26 ms │              29.13 ms │ +1.07x faster │
│ QQuery 17 │ 159.46 ms │             154.29 ms │     no change │
│ QQuery 18 │ 287.59 ms │             284.81 ms │     no change │
│ QQuery 19 │  39.20 ms │              39.99 ms │     no change │
│ QQuery 20 │  56.56 ms │              57.03 ms │     no change │
│ QQuery 21 │ 300.14 ms │             325.49 ms │  1.08x slower │
│ QQuery 22 │  22.64 ms │              22.18 ms │     no change │
└───────────┴───────────┴───────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 1858.59ms │
│ Total Time (alamb_update_arrow_58)   │ 1864.00ms │
│ Average Time (HEAD)                  │   84.48ms │
│ Average Time (alamb_update_arrow_58) │   84.73ms │
│ Queries Faster                       │         3 │
│ Queries Slower                       │         2 │
│ Queries with No Change               │        17 │
│ Queries with Failure                 │         0 │
└──────────────────────────────────────┴───────────┘

alamb-ghbot · 2026-01-10T19:18:26Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_58 (35b97fa) to b9a3b9f diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-01-10T19:19:04Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_58
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ alamb_update_arrow_58 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 200.13 ms │             195.58 ms │     no change │
│ QQuery 2  │ 101.86 ms │             103.65 ms │     no change │
│ QQuery 3  │ 128.67 ms │             121.90 ms │ +1.06x faster │
│ QQuery 4  │  78.40 ms │              77.50 ms │     no change │
│ QQuery 5  │ 172.69 ms │             172.92 ms │     no change │
│ QQuery 6  │  65.58 ms │              68.66 ms │     no change │
│ QQuery 7  │ 219.12 ms │             245.34 ms │  1.12x slower │
│ QQuery 8  │ 175.19 ms │             177.33 ms │     no change │
│ QQuery 9  │ 227.09 ms │             244.95 ms │  1.08x slower │
│ QQuery 10 │ 184.88 ms │             190.38 ms │     no change │
│ QQuery 11 │  80.82 ms │              78.24 ms │     no change │
│ QQuery 12 │ 117.62 ms │             116.55 ms │     no change │
│ QQuery 13 │ 223.12 ms │             215.42 ms │     no change │
│ QQuery 14 │  93.28 ms │              93.53 ms │     no change │
│ QQuery 15 │ 128.30 ms │             123.42 ms │     no change │
│ QQuery 16 │  59.98 ms │              60.86 ms │     no change │
│ QQuery 17 │ 270.35 ms │             272.73 ms │     no change │
│ QQuery 18 │ 316.62 ms │             314.87 ms │     no change │
│ QQuery 19 │ 131.64 ms │             137.17 ms │     no change │
│ QQuery 20 │ 132.77 ms │             130.38 ms │     no change │
│ QQuery 21 │ 271.59 ms │             274.52 ms │     no change │
│ QQuery 22 │  47.46 ms │              48.44 ms │     no change │
└───────────┴───────────┴───────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                    ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                    │ 3427.15ms │
│ Total Time (alamb_update_arrow_58)   │ 3464.32ms │
│ Average Time (HEAD)                  │  155.78ms │
│ Average Time (alamb_update_arrow_58) │  157.47ms │
│ Queries Faster                       │         1 │
│ Queries Slower                       │         2 │
│ Queries with No Change               │        19 │
│ Queries with Failure                 │         0 │
└──────────────────────────────────────┴───────────┘

alamb added 10 commits January 6, 2026 17:55

Update to pre-release arrow

120d2b0

Fix clippy due to UnionFields::new is deprecated

40a6a71

Fix real bug in test

f50b9c3

Update test for fixed issue upstream

87563d8

Update tests for new feature

4afd46f

Update tests for struct casting

70e2841

Update to latest

b29b662

Merge remote-tracking branch 'apache/main' into alamb/update_arrow_57…

6a2e631

….2.0

Merge branch 'main' into alamb/update_arrow_57.2.0

ca11966

Update to pre-release main

63a85eb

github-actions bot added sqllogictest SQL Logic Tests (.slt) common Related to common crate proto Related to proto crate functions Changes to functions implementation datasource Changes to the datasource crate physical-plan Changes to the physical-plan crate labels Jan 10, 2026

Update expected value of make_interval

277aac4

alamb commented Jan 10, 2026

View reviewed changes

Update expected output

bb29b0c

alamb commented Jan 10, 2026

View reviewed changes

alamb mentioned this pull request Jan 10, 2026

feat: change default behavior for Parquet PageEncodingStats to bitmask apache/arrow-rs#9051

Merged

Update to latest arrow

35b97fa

alamb mentioned this pull request Jan 10, 2026

fix: liberal parsing of zero scale decimals apache/arrow-rs#8700

Open

WIP: Upgrade DataFusion to arrow-rs/parquet 58.0.0 #19728

Are you sure you want to change the base?

WIP: Upgrade DataFusion to arrow-rs/parquet 58.0.0 #19728

Conversation

alamb commented Jan 10, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

davidlghellin Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jefffrey Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

alamb Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Dandandan commented Jan 10, 2026

Uh oh!

alamb-ghbot commented Jan 10, 2026

Uh oh!

alamb-ghbot commented Jan 10, 2026

Uh oh!

Dandandan commented Jan 10, 2026

Uh oh!

Dandandan commented Jan 10, 2026

Uh oh!

alamb-ghbot commented Jan 10, 2026

Uh oh!

alamb-ghbot commented Jan 10, 2026

Uh oh!

alamb-ghbot commented Jan 10, 2026

Uh oh!

alamb-ghbot commented Jan 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants