Context: apache/iceberg-rust#2442 (comment)
pyiceberg[pyarrow] currently installs both pyarrow and pyiceberg-core. This is too broad: most Arrow read/write paths only need pyarrow, while pyiceberg-core is only used in lazy, feature-specific code paths (native transform kernels, DataFusion table provider).
It also creates a packaging cycle for downstream projects: pyiceberg-core cannot depend on pyiceberg[pyarrow] without re-installing itself through the extra.
Context: apache/iceberg-rust#2442 (comment)
pyiceberg[pyarrow]currently installs bothpyarrowandpyiceberg-core. This is too broad: most Arrow read/write paths only needpyarrow, whilepyiceberg-coreis only used in lazy, feature-specific code paths (native transform kernels, DataFusion table provider).It also creates a packaging cycle for downstream projects:
pyiceberg-corecannot depend onpyiceberg[pyarrow]without re-installing itself through the extra.