List view

Is it possible to support PyArrow backed UDFs in Comet natively?

enhancementNew feature or requestNew feature or request

Status: Open.

#957 In apache/datafusion-comet;

· SemyonSinchenko opened on Sep 21, 2024

Register CometArrowAllocator as a Spark MemoryConsumer for JVM-UDF dispatch

area:expressionsExpression evaluationExpression evaluation

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4174 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Make JVM-scalar-UDF dispatch responsive to task cancellation

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4175 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Add metrics and logging to JVM-scalar-UDF dispatch path

area:expressionsExpression evaluationExpression evaluation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4176 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Add support for scalar UDFs that operate on Arrow data

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4177 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Support higher-order array functions via JVM UDF bridge

area:expressionsExpression evaluationExpression evaluation

area:udf

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4224 In apache/datafusion-comet;

· andygrove opened on May 5, 2026

Eliminate remaining row<->Arrow round-trip in CometPythonMapInArrowExec

area:udf

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4240 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

Disable Comet if shuffle manager not registered

enhancementNew feature or requestNew feature or request

performance

Status: Open.

#3900 In apache/datafusion-comet;

· andygrove opened on Apr 3, 2026

[Research] Use custom cost model when deciding between SMJ and SHJ

enhancementNew feature or requestNew feature or request

performance

Status: Open.

#1011 In apache/datafusion-comet;

· andygrove opened on Oct 10, 2024

Add native implementations ofRegExpExtract and RegExpExtractAll

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

good first issueGood for newcomersGood for newcomers

Status: Open.

#2708 In apache/datafusion-comet;

· andygrove opened on Nov 6, 2025

Improve performance of aggregate expressions by implementing group accumulators

enhancementNew feature or requestNew feature or request

performance

Status: Open.

#1627 In apache/datafusion-comet;

· andygrove opened on Apr 8, 2025

[EPIC] Improve performance of TPC-DS queries

enhancementNew feature or requestNew feature or request

performance

Status: Open.

#858 In apache/datafusion-comet;

· andygrove opened on Aug 21, 2024

Re-enable COUNT for mixed Spark partial / Comet final aggregate execution

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4242 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

Various issues with Comet support for HashAggregateExec

enhancementNew feature or requestNew feature or request

Status: Open.

#1267 In apache/datafusion-comet;

· andygrove opened on Jan 10, 2025

Improve fallback reporting for aggregates

enhancementNew feature or requestNew feature or request

Status: Open.

#2787 In apache/datafusion-comet;

· andygrove opened on Nov 14, 2025

Bloom filter intermediate aggregate buffers are not compatible between Spark and Comet

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

bugSomething isn't workingSomething isn't working

crashNative engine crash/panic/segfaultNative engine crash/panic/segfault

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#2889 In apache/datafusion-comet;

· andygrove opened on Dec 12, 2025

[EPIC] Improve planning of hash aggregates

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

bugSomething isn't workingSomething isn't working

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#2892 In apache/datafusion-comet;

· andygrove opened on Dec 12, 2025

Comet falls back to Spark for final hash aggregate in some cases when it could be supported

enhancementNew feature or requestNew feature or request

Status: Open.

#2894 In apache/datafusion-comet;

· andygrove opened on Dec 13, 2025

Improve performance of first/last aggregates

enhancementNew feature or requestNew feature or request

performance

Status: Open.

#3022 In apache/datafusion-comet;

· andygrove opened on Jan 2, 2026

Support SubqueryBroadcastExec in Comet

enhancementNew feature or requestNew feature or request

performance

Status: Open.

#242 In apache/datafusion-comet;

· viirya opened on Apr 5, 2024

Mismatch in Data Types Between Partial and Final Stages in Aggregations

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

bugSomething isn't workingSomething isn't working

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#1287 In apache/datafusion-comet;

· andygrove opened on Jan 14, 2025

AQE may materialize a non-supported Final-mode HashAggregate

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

bugSomething isn't workingSomething isn't working

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#1389 In apache/datafusion-comet;

· EmilyMatt opened on Feb 12, 2025

Add tests for map types to CometFuzzTestSuite

enhancementNew feature or requestNew feature or request

Status: Open.

#1665 In apache/datafusion-comet;

· andygrove opened on Apr 21, 2025

native_iceberg_compat uses hard-coded config values

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_iceberg_compatSpecific to native_iceberg_compat scan typeSpecific to native_iceberg_compat scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#1816 In apache/datafusion-comet;

· parthchandra opened on May 29, 2025

Add support for size expression

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

Status: Open.

#1926 In apache/datafusion-comet;

· andygrove opened on Jun 23, 2025